S3(Swift, COS, OSS) Content Review
Introduces how to call the interface to review S3 content in the WebService application channel and asynchronous mode. This step also applies to cloud content services such as Swift, COS, and OSS. For the specific supported cloud service providers, refer to the section objectinfo Parameters for Reviewing Cloud Service Stored Content in Asynchronous Mode.
Batch Review
Introduces calling the WebService application channel to send a JSON text via POST for batch content review. The Content-type in the HTTP header needs to be set to "application/json". This article takes batch review of S3 content as an example.Request method
POST
URL
Request URL
/skg/v1/dlp/channel/webserviceapp/<WebService Application ID>/bulk_async
Request Parameters
| Parameter Name | Type/Required | Description |
|---|---|---|
| callback_url | String (Required) | The callback URL for asynchronous mode, which will be called after all object checks are completed. |
| verbose | Bool (Required) | Whether to return incident_info. |
| objects | List (Required) | List of objects to be checked, for specific parameters refer to the table below. |
The following table provides detailed information about the parameters of the objects list.
| Parameter Name | Type | Description |
|---|---|---|
| user | String (Required) | The username that generated the event - supports domain users, in the format domain\username. |
| filename | String (Optional) | File name - can be used for policy matching based on file name |
| customAttribute | String (Optional) | Supports user-defined parameter names. |
| queryID | String (Required) | The event query ID associated with this request, must be unique. If no event is generated for the request, the event details cannot be queried. Note: The value of queryid corresponds to the traffic UUID in the third-party cloud service. |
| callback_url | String (Optional) | The listening address for the callback |
| verbose | String (Required) | Whether to return event information. |
| uploadtype | String (Required) | S3, the supported upload type for batch upload is S3. Note: If it is other cloud content, change it to the corresponding cloud content type. The following cloud content types are currently supported:
|
| s3Region | String (Required) | The region name when using s3, the following three items can be omitted if running on EC2 and using AWS IAM role.
Note: This parameter, as well as the following accessKey, secretKey, endpointUrl, s3Bucket, and s3Key are all S3-related parameters. For parameters of other cloud services such as Swift and OSS, refer to the section objectinfo Parameters for Reviewing Cloud Service Stored Content in Asynchronous Mode. |
| accessKey | String (Required) | The access key when using s3. |
| secretKey | String (Required) | The secret key when using s3. |
| endpointUrl | String (Required) | The endpoint url when using s3. |
| s3Bucket | String (Required) | The object bucket when using s3 |
| s3Key | String (Required) | The object key (file name) when using s3 |
Python Request Example - Batch Submission
The following example shows how to use Python code to call the interface to upload files stored in S3 in batches for content security review.
import json
import requests
auth_headers = {
"Authorization": "<Authorization>",
"x-skg-timestamp": "<timestamp>",
}
ucwi_endpoint = "<ENDPOINT>"
url = ucwi_endpoint + "/skg/v1/dlp/channel/webserviceapp/78217a43-542d-47e2-97a5-4435bf11521d/bulk_async"
data = {
"callback_url": CALLBACK_URL,
"verbose": False,
"objects": [
{
"user": "cloudtest\\clouduser",
"filename": "file1",
"queryID": "<queryID2>",
"uploadtype": "s3",
"objectInfo": {
"s3Region": "cn-north-1",
"accessKey": "<AWS_AK>",
"secretKey": "<AWS_SK>",
"endpointUrl": "https://s3.cn-north-1.amazonaws.com.cn",
"s3Bucket": "falseuser-test1",
"s3Key": "Desert1.jpg",
},
{
"user": "cloudtest\\clouduser",
"filename": "file1",
"queryID": "<queryID3>",
"uploadtype": "s3",
"objectInfo": {
"s3Region": "cn-north-1",
"accessKey": "<AWS_AK>",
"secretKey": "<AWS_SK>",
"endpointUrl": "https://s3.cn-north-1.amazonaws.com.cn",
"s3Bucket": "falseuser-test1",
"s3Key": "Deser2.jpg",
},
}
]
}
r = requests.post(url, headers=auth_headers, json=data)
print(r.json())
Return Parameters
The batch review request return results include the following parameters:
| Name | Description |
|---|---|
| result | Whether the request was successful, 0 for success, 1 for failure |
| actionCode | When the request is successful, users can choose to perform default operations on content that meets the request conditions, including 1 - allow data transfer and 2 - block data transfer |
| errorCode | The error code returned when the request fails |
| message | The error message returned when the request fails |
| Incident_info | Event information for policy matches. If a policy is matched and non-compliant content is found, the system will also return policy match information. Refer to the return example for details. |
Return Example
If a policy is matched and non-compliant content is found, the system will also return policy match information.
The following example and its notes provide detailed information about each parameter in the return results and its meaning.
{
"cd2fd109-c4d4-489f-9b27-53752f7827d": {
"localDetectedTime": "2019-07-29T16:17:43.079368+0800", //Detection Time
"incident_info": //Incident Information
{
"matchedPolicies": [ //Matched Policies
{
"numberOfMatches": 1, //Number of Events for Matched Policies
"name": "abc\\enduser1", //Domain and Username of the Violating User
"actionSettingName": "Block", //Action Executed for Violations
"matchedRules": [ //Matched Policy Rules
{
"name": "company", //Rule Name
"matchedConditions": [ //Matched Conditions
{
"type": 5, //Condition Match Type, including: 1:Regular Expression, 2:Dictionary, 3:External Script, 4:File Type Group, 5:Keyword, 6:Script, 7:File Fingerprint, 8:Machine Learning, 9:Terminal Location, 10:File Name, 11:Attachment Quantity, 12:Database Fingerprint, 13:File Size, 14:Binary, 15:Compressed File Depth, 16:Encrypted File, 17:Format Mismatch File, 18:Built-in Template
"matchedElements": [ //Matched Elements
{
"matchedContents": [ //Matched Content
{
"detectedValues": [ //Detected Content
{
"text": "company" //Keyword
}
],
"isFileSuffixMatch": true, //Whether the file suffix matches
"isArchiveFile": false, //Whether it is a compressed file
"isEncryptFile": false, //Whether it is an encrypted file
"encodeType": "UnknownEncoding", //File encoding type, for details, refer to international standard encoding types and their explanations
"numberOfMatches": 1, //Number of Matches
"locationPath": "confidential.doc", //Full file path
"id": "0-0", //File ID
"contentSize": 22 //Content Size
}
],
"numberOfMatches": 1,
"isTruncated": false
}
],
"isTraditionalMatching": false //Whether traditional matching is enabled
}
]
}
],
"priority": 31, //Priority
"groupName": "Default Policy Group", //Policy Group Name
"severity": 3 //Sensitivity Level 1 - High, 2 - Medium, 3 - Low, 4 - Information
}
]
},
"result": 0,
"actionCode": 2
},
"47992d42-04b7-4860-b186-b8c11f8b2253": {
"localDetectedTime": "2019-07-29T07:52:26.423684",
"incident_info": {},
"result": 0,
"actionCode": 1
}
}