Typical Deployment - Big Data Platform

Introduces information about the deployment of big data platforms.

Supports typical deployments for big data processing platforms, including the following deployment scenarios:

• One of the data processing applications, the data collector, collects data from data generators and stores the raw data in a data storage. Another application (data cleaner) removes sensitive information or edits it, and stores the cleaned data in another data storage.
• This deployment assumes that only a few privileged users in the office can access the raw data, while other office users can only access the cleaned data. In addition, the platform policy stipulates that data containing geographical location information, even after processing, should not be released to non-member data users. However, since all applications can access both data storages, it becomes the burden of application developers to comply with the above rules when processing data. Once there is an error in the application logic, non-privileged users may access some unprocessed raw data for a long time without triggering a warning, resulting in some delivered application data containing geographical location information of external user data due to the inability to identify geographical location information.
• With the deployment of SkyGuard™Unified Content Web-Service Inspector (UCWI), the platform data security team creates and maintains all security policies on the Unified Content Secure Server (UCSS), and the team automatically publishes these policies to Unified Content Web-Service Inspector (UCWI). Application developers only need to submit data with appropriate APIs at key data transmission points to ensure that inappropriate data is not passed to the wrong recipients. Therefore, over a period of time, repeated Data Loss Prevention (DLP) event warnings will be received. When data is passed to privileged users, sensitive data will be blocked. Developers realize that there are errors in their code and that they are reading data from the raw data storage when presenting data to privileged users. In addition, when application developers send data to external users, they can use APIs to filter out data containing geographical location information, thus eliminating the possibility of programming errors.
• SkyGuard™Unified Content Web-Service Inspector (UCWI) supports the HA device group mode, which allows two or more Unified Content Web-Service Inspector (UCWI) devices to be set up as a device group and registered to Unified Content Secure Server (UCSS). The devices in the high-availability device group serve as backups for each other. When any device in the high-availability group fails, other devices in the group can immediately take over to ensure normal system service operation without interruption.
  Important: When a device experiences hardware failure (power outage, network card issues, etc.), network failure (network cable connection issues, other network device failures), etc., device switching cannot be achieved.
  Note: The system supports high-availability device switching within 10 seconds to ensure high availability, while also ensuring the synchronization of logs, events, evidence, and configurations without loss.