POST-Content Review-Web Channel

Interface Introduction

The Web Channel Content Review interface is used to detect whether the shared files uploaded by users in the Web channel violate the enterprise or organization's Data Loss Prevention (DLP) policies and to perform response actions such as allowing or blocking data transmission.

Request method

Interface URL

Request Parameters

Definition of Body parameters in the request.

Request Examples

The following examples show how to call the interface to send data to Unified Content Web-Service Inspector (UCWI) for content security inspection in both synchronous and asynchronous modes.

• Synchronous Mode:

  POST /skg/v1/dlp/channel/web/http/sync
  Content-Type: multipart/form-data; boundary=${bound}
  --${bound}
  Content-Disposition: form-data; name="metadata"
  Content-Type: application/json
  {
  "sourceAddress":"123.12.12.112",
  "user":"abc\enduser1",
  "url":"http://www.sina.com.cn/forum/post.action",
  "httpBodyLen" : 10240,
  "queryID" : "1024f306-566a-415f-9ada-1be89b9f1086",
  "httpHeader":"Host: testhost.com\r\nConnection: Keep-Alive\r\nContent-Type: text/plain\r\nContent-Length:000000010240\r\n\r\n",
  "callback_url":"http://172.22.113.49:5000/post/http"
  }
  --${bound}
  Content-Disposition: form-data; name="request"; filename="httpbody"
  Content-Type: application/octet-stream
  %HTTP BODY%

  HTTP BODY format:

  Film and the City: The Urban Imaginary in Canadian Cinema.

• Asynchronous Mode:

  POST /skg/v1/dlp/channel/web/http/async
  Content-Type: multipart/form-data; boundary=${bound}
  --${bound}
  Content-Disposition: form-data; name="metadata"
  Content-Type: application/json
  {
  "sourceAddress":"123.12.12.112",
  "user":"abc\enduser1",
  "url":"http://www.sina.com.cn/forum/post.action",
  "httpBodyLen" : 10240,
  "queryID" : "1024f306-566a-415f-9ada-1be89b9f1086",
  "sendBack": {
          "type": "s3",
          "s3Bucket": "test",
          "s3Region": "", # Upload the redacted file to this location. The value here is the region name when using S3.
  # If running on EC2 and using AWS IAM role, the following three items can be omitted, then the following three accessKey, secretKey, endpointUrl can be omitted
          "accessKey": "aws_accessKey",
          "secretKey": "aws_secretKey",
          "endpointUrl": ""
    },
  "redaction": {
          "enable": true # The request takes effect when enable is true
    }
  "httpHeader":"Host: testhost.com\r\nConnection: Keep-Alive\r\nContent-Type: text/plain\r\nContent-Length:000000010240\r\n\r\n",
  "uploadtype":"file/s3",
  "callback_url":"http://172.22.113.49:5000/post/http"
  }
  --${bound}
  Content-Disposition: form-data; name="request"; filename="http://s3_url/httpchannel/http/http.txt"
  Content-Type: application/octet-stream
  %HTTP BODY%

Python Request Example - HTTP Channel (Synchronous Mode)

The following example shows how to use Python code to call the interface to send files in the HTTP channel for content security review in synchronous mode.

# -*- coding: utf-8 -*-
from requests.packages.urllib3.exceptions import InsecureRequestWarning
from ucwi_config import UCWIConfig
from ucwi_auth import get_headers
import requests
import json
import os
import uuid

requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

# protocol value: "http" "https"
protocol = "http"
api = "/skg/v1/dlp/channel/web/{}/sync".format(protocol)
url = "{0}{1}".format(UCWIConfig.base_url, api)
file_path = "test.txt"

metadata = {
    "user": "ucwitestuser",
    # The user who sends the data, used for policy matching during content security review, and for event display after content security review is completed
    "sourceAddress": "172.22.1.1",
    # The source IP address of the data, used for policy matching during content security review, and for event display after content security review is completed
    "httpBodyLen": str(os.path.getsize(file_path)),
    # The size of the HTTP packet body being sent, used for policy matching during content security review
    "queryID": str(uuid.uuid4()),
    # The UUID associated with this request, must be unique, used for event queries in asynchronous mode
    "httpHeader": "Host: 127.0.0.1\r\nConnection: Keep-Alive\r\nContent-Type: text/plain\r\nContent-Length: 10240\r\n\r\n",
    # The Header of the HTTP packet sent by the user, i.e., the data packet header information
    "url": "http://172.16.0.1/post.php",
    # The URL address of the HTTP request
}

headers = get_headers()
data = {"metadata": json.dumps(metadata)}
fd = open(file_path)
files = {
    "request": fd
}
response = requests.post(url, headers=headers, data=data, files=files, verify=False)
fd.close()

if response.status_code != 200:
    print("Bad request, response code:", response.status_code)
    print(response.text)
else:
    result = response.json()
    # print(json.dumps(result, indent=4).decode('raw_unicode_escape'))
    if result["responseCode"] != 200:
        print("Bad request, response code:", result["responseCode"])
        print(result["message"])
    else:
        hint = "# 1: Allow; 2: Block; 3: Confirm; 4: Delete Attachment; 5: Email Encryption; 6: Email Quarantine; 7: Terminal System Encryption; 8: Email Content Encryption; 9: Terminal Personal Key Encryption"
        print("action:{}    {}".format(result["actionCode"], hint))
        if len(result["incident_info"]) == 0:
            print("not matched.")
        else:
            print(result)
            print("matched policy:")
            for policy in result["incident_info"]["matchedPolicies"]:
                print(json.dumps(policy, indent=2).encode('utf-8').decode('raw_unicode_escape'))

Python Request Example - HTTP Channel (Asynchronous Mode)

The following example shows how to use Python code to call the interface to send files in the HTTP channel for content security review in asynchronous mode.

# -*- coding: utf-8 -*-
from requests.packages.urllib3.exceptions import InsecureRequestWarning
from ucwi_config import UCWIConfig
from ucwi_auth import get_headers
import requests
import json
import uuid
import os

requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

# protocol value: "http" "https"
protocol = "http"
api = "/skg/v1/dlp/channel/web/{}/async".format(protocol)
url = "{0}{1}".format(UCWIConfig.base_url, api)
file_path = "test.txt"

metadata = {
    "user": "ucwitestuser",
    # The user who sends the data, used for policy matching during content security review, and for event display after content security review is completed
    "sourceAddress": "172.22.1.1",
    # The source IP address of the data, used for policy matching during content security review, and for event display after content security review is completed
    "httpBodyLen": str(os.path.getsize(file_path)),
    # The size of the HTTP packet body being sent, used for policy matching during content security review
    "queryID": str(uuid.uuid4()),
    # The UUID associated with this request, must be unique, used for event queries in asynchronous mode
    "httpHeader": "Host: 127.0.0.1\r\nConnection: Keep-Alive\r\nContent-Type: text/plain\r\nContent-Length: 10240\r\n\r\n",
    # The Header of the HTTP packet sent by the user, i.e., the data packet header information
    "url": "http://172.16.0.1/post.php",
    # The URL address of the HTTP request
    "uploadtype": "file",
    "callback_url": UCWIConfig.callback_url + "/web"
}

headers = get_headers()
data = {"metadata": json.dumps(metadata)}
fd = open(file_path)
files = {
    "request": fd
}
response = requests.post(url, headers=headers, data=data, files=files, verify=False)
fd.close()

if response.status_code != 200:
    print("Bad request, response code:", response.status_code)
    print(response.text)
else:
    result = response.json()
    print(result["message"])

Java Request Example - HTTP Channel (Synchronous Mode)

The following example shows how to use Java code to call the interface to send files in the HTTP channel for content security review in synchronous mode.

import java.util.HashMap;
import java.util.Iterator;
import java.util.Properties;
import java.util.UUID;
import java.io.File;
import java.io.FileInputStream;

import javax.net.ssl.SSLContext;
import java.security.cert.X509Certificate;

import com.alibaba.fastjson.JSONObject;
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.entity.mime.content.StringBody;
import org.apache.http.util.EntityUtils;

public class TestHttp {

    public static String inspectHttpFile(File file) {

        String host = "<UCWI_HOST>";
        String port = "<UCWI_PORT>";
        String apiPath = "/skg/v1/dlp/channel/web/http/sync";
        String url = "https://" + host + ":" + port + apiPath;

        String queryId = UUID.randomUUID().toString();

        JSONObject metadataJson = new JSONObject();
        metadataJson.put("user", "cloudtest\\httpuser");
        metadataJson.put("sourceAddress", "172.22.117.242");
        metadataJson.put("httpBodyLen", "987111");
        metadataJson.put("queryID", queryId);
        metadataJson.put("httpHeader", "Host: testhost.com");
        metadataJson.put("url", "http://172.16.0.1/post.php");

        HttpPost post = new HttpPost(url);

        post.addHeader("Authorization", "<Authorization>");
        post.addHeader("x-skg-timestamp", "<timestamp>");

        CloseableHttpClient client = HttpClients.createDefault();
        try {
            SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy() {
                public boolean isTrusted(X509Certificate[] chain, String authType) {
                    return true;
                }
            }).build();
            SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext, new NoopHostnameVerifier());
            client = HttpClients.custom().setSSLSocketFactory(sslsf).build();
        }
        catch (Exception e){
            e.printStackTrace();
        }

        String metadataString = metadataJson.toJSONString();

        MultipartEntityBuilder builder = MultipartEntityBuilder.create();
        StringBody metadataPart = new StringBody(metadataString, ContentType.TEXT_PLAIN);
        builder.addPart("metadata", metadataPart);

        FileBody filePart = new FileBody(file);
        builder.addPart("request", filePart);

        HttpEntity multipart = builder.build();
        post.setEntity(multipart);
        String result = "";
        try {
            CloseableHttpResponse resp = client.execute(post);
            HttpEntity entity = resp.getEntity();
            result = EntityUtils.toString(entity, "UTF-8");
        }
        catch (Exception e){
            e.printStackTrace();
        }

        return result;
    }

    public static void main(String []args) {
        String filepath = "/tmp/file1";
        File f = new File(filepath);
        String result = inspectHttpFile(f);
        System.out.println(result);
    }
}

Obtaining Authentication Information

Use the following example with cURL to obtain the key information required for authentication.

cURL Request Example - HTTP Channel (Asynchronous Mode)

Use the following example to call the interface for content review in the Web channel using cURL.

Return Parameters

Including the following：

Example for return code

The response to the Content inspection request is as follows.

• Request successfully

  {
  "result" : 0,
  "actionCode" : 1/2
  }
  actionCode: 1 - allow, 2 - block

  Note: If the policy is matched and the violation content is found, the system returns the policy matching information. Refer to The return value for policy match.
• Request failed

  {
  "result" : 1,
  "errorCode" : 500,
  "message" : "Invalid parameter"
  }

Error code