What Adani port actually is
The core business of Adani Ports or APSEZ in short revolves around developing and operating port infrastructure and providing a comprehensive range of port services. This can be broken down into several key areas:
1 Cargo Handling: This is the fundamental activity of APSEZ, involving the loading and unloading of various cargo types, including containers, breakbulk cargo. Efficiency in cargo handling is crucial for minimizing vessel turnaround times and optimizing port capacity. APSEZ invests heavily in modern equipment and technology, such as automated cranes and terminal operating systems, to enhance their cargo handling capabilities.
2 Storage and Warehousing: APSEZ provides storage facilities for various cargo types within their port areas. These facilities include warehouses, tank farms for liquid bulk, and specialized storage areas for hazardous materials. Secure and efficient storage is essential for maintaining cargo integrity and facilitating smooth logistics operations.
3 Pilotage and Towage: These marine services are critical for guiding vessels safely in and out of the port and assisting with berthing and unberthing operations. APSEZ provides these services through its fleet of tugs and experienced pilots, ensuring the safe and efficient movement of vessels within the port.
4 Container Terminal Operations: A significant portion of APSEZ's business focuses on container handling. They operate dedicated container terminals equipped with SOTA infrastructure and technology, including gantry cranes, container handling equipment, and advanced terminal management systems. Efficient container handling is crucial for facilitating global trade and ensuring the smooth flow of goods.
5 Marine Services and Infrastructure Development: APSEZ is involved in developing and maintaining essential port infrastructure, including berths, jetties, breakwaters, and navigation channels. They also provide a range of marine services to support vessel operations, such as dredging, bunkering, and ship repairs. Continuous investment in infrastructure development is crucial for expanding port capacity and accommodating larger vessels.
Codelinter was tasked to perform data classification within point#4: Terminal Operations
While main business aspects covered from point#1 to point#5 are all interconnected. Codelinter was specifically chosen to handle the message auditing within the control plane of the operations. APSEZ handles all operations via this control plane and is always available 24/7. This department is important enough to give the bird's view of everything that is happening within the dock yard.
While Codelinter is not allowed to specifically talk about the core technology of operations of these terminals since they are deemed senstive information by APSEZ. But we will go through some key points of this management plane at APSEZ.
1 Reports:
Every bulk carrier, container, cargo and tanker is recognized and tagged using their proprietary AIS systems. Their terminal CP (control plane) is designed to locate and identify them in near real time. The very important aspect of terminal CP is to be able to make live visual reports of every tagged hardware. In APSEZ's terms we communicate with over 8000 such hardware every hour. Out of which around 2500 are tracking sensors. The report includes various Ro-Ro points using their Electronic Data Interchange (EDI) artifacts.
2 Classification:
Terminal CP is designed to gather all wayland points within the anchorage port yards. However, not all containers have identical EDI blueprints since some of them were built in old times and a lot of times they differ from continent to continent. So Codelinter was tasked to design software that allowed APSEZ to easily and conveniently perform EDI data classification in real time. Because the requirement was to perform internal storage of all EDI artifacts, Codelinter chose Nats jetstream for the audit storage which allowed APSEZ managers access to replayable write ahead logs style events of EDI classification pipeline.
NATS to the rescue
So, what is NATS?
NATS is, simply, a highly performant and reliable messaging system that addresses critical challenges faced by high throughtput industries with an extra ability to store messages in encrypted form and to be able to easily replay them.
According to nats.io
Nats is a single technology that enables applications to securely communicate across any combination of cloud vendors, on-premise, edge, web and mobile, and devices
The above description really holds true indeed. In order to generate and transfer reports in realtime that would feed into the classification pipeline, Codelinter used nats core to manage real time communication among 15 microservices on 150 devices each deployed within the periphery of the dock yard terminal zones allowing secure and efficient data transfers in real time, while using less than 10% CPU. Most devices were embedded devices with 2Ghz less than or equal horse power.
To collect, evaluate and replay audit logs we used NATS jetstream in cluster mode. Jetstream allowed to perform encrypt messages at rest using chachapoly cipher (the current SOTA encryption).
We also forked nats cli, among other auxilliary libraries, to incorporate encryption of all the config artifacts generated by out custom deployment toolset using the amazing nsc binary. This made config deployment tamper resistant.
Conclusion:
NATS proved to be a very lightweight and robust technology to perform service to service communication especially between polyglot services. Jetstream turned out to be a good fit when came to replayable logs. The only grype with cluster mode jetstream with n-n node replication enabled is that if one of the nodes go down, it stops taking any more updates and goes into read only mode. This is something we need to look further to make it more fault tolerant.
