Ensure User Privacy with Consent Management Platform (CMP)
Project Summary
The objective was to build a Consent Management Platform (CMP) that will enable the customers to adhere to the GDPR-compliance, reduce the privacy risk levels, and ensure risk-free marketing. This sophisticated CMP software would be able to provide visits to its customers.
Technologies Used-
- Node.js
- GraphQL
- MongoDB
- Redis
- Docker
- Terraform
- Google File Storage
- Google Cloud Services for deployment
- Bigtable and BigQuery for Database
- Automated deployment
- Fully managed, flexible, and scalable server capable to manage the huge traffic
Challenges
-
To develop the consent management platform, the primary step was to build a widget that would include the text, images, language, and other settings to enable the users to accept cookies as per their preferences.
-
The CMP development required a sophisticated database to manage the incoming API requests, a standardized approach for data optimization and report generation, and scalable server solutions.
-
Traffic Management was a challenging factor as the huge traffic rush caused a blockage in the flow, which in turn crashed the server. To resolve it, we used a Stack driver logger to manage requests and trace them.
-
Data optimization and metadata management was, again, a challenging task as the scenarios were wide and entangled together. Each customer had a setting ID, and it was complicated to consider every angle as to what to showcase in what respect, how to generate a report for a particular customer, and optimizing the data at the same time.
To develop the consent management platform, the primary step was to build a widget that would include the text, images, language, and other settings to enable the users to accept cookies as per their preferences.
The CMP development required a sophisticated database to manage the incoming API requests, a standardized approach for data optimization and report generation, and scalable server solutions.
Traffic Management was a challenging factor as the huge traffic rush caused a blockage in the flow, which in turn crashed the server. To resolve it, we used a Stack driver logger to manage requests and trace them.
Data optimization and metadata management was, again, a challenging task as the scenarios were wide and entangled together. Each customer had a setting ID, and it was complicated to consider every angle as to what to showcase in what respect, how to generate a report for a particular customer, and optimizing the data at the same time.
Solutions
-
“Initially, the database used for the project was Google data storage. However, events turned challenging after launch as we started receiving massive requests, up to 2000 request updates per second. Considering the initial requests, which were about 5 update requests per second, the data we received was huge and could not be handled through an existing database.
-
The performance and scalability were at stake and required an immediate switch to a better database. We thoroughly studied all the scenarios and switched to MongoDB. The code we used was reusable, maintainable, and loosely coupled with that.
-
The platform had multiple tools and features with numerous scenarios that added complexity to it. For instance, translation itself was multifaceted as we were integrated into 17+ languages. We were using DeepL Translator for translation, however, DeepL was an expensive tool, and our primary criterion was to reduce the project cost.
-
The update requests we received were in thousands and the translation storage was dynamic as multiple users could select any language at the same time and some languages were among the languages that we were not integrated with.
-
To get ahead of these challenges, Team Habilelabs conducted regular testing, implemented agile methods, and consulted with the client to receive their regular feedback about the ongoing developments.”
“Initially, the database used for the project was Google data storage. However, events turned challenging after launch as we started receiving massive requests, up to 2000 request updates per second. Considering the initial requests, which were about 5 update requests per second, the data we received was huge and could not be handled through an existing database.
The performance and scalability were at stake and required an immediate switch to a better database. We thoroughly studied all the scenarios and switched to MongoDB. The code we used was reusable, maintainable, and loosely coupled with that.
The platform had multiple tools and features with numerous scenarios that added complexity to it. For instance, translation itself was multifaceted as we were integrated into 17+ languages. We were using DeepL Translator for translation, however, DeepL was an expensive tool, and our primary criterion was to reduce the project cost.
The update requests we received were in thousands and the translation storage was dynamic as multiple users could select any language at the same time and some languages were among the languages that we were not integrated with.
To get ahead of these challenges, Team Habilelabs conducted regular testing, implemented agile methods, and consulted with the client to receive their regular feedback about the ongoing developments.”
“It was, on the whole, a joyful and learning experience for the entire team. We learned about the sensitive issue of digital privacy as data was being collected through multiple trackers in the form of third-party cookies. The CMP protects the data privacy of the user and enables them to selectively concur as to what data can be tracked and to what ends. We were glad to be a part of the project being designed to ensure safe browsing and data privacy.”
