Aggregation and processing of big data and distribution of open data

The main objective of our activity is to design and implement:

  • a model of acquisition, analysis and validation to support city-level data planning and management;
  • an open platform for aggregating and presenting open data.

In particular, for the data aggregation, a three-layer lambda architecture will be implemented, consisting of:

  • a batch layer for periodic processing;
  • a server layer, based on a scalable database, to reorganize information generated by batch processing to make it accessible in an efficient way by the remaining components of the system;
  • a speed layer for online analysis of incoming data to the processing system, compensating for the high latency of the serving layer updates.

The system will be specialized and interfaced with both data generators and project sensors (radars, low cost sensors, mobile sensors) and pre-existing data sources such as municipal sensors (weather and area quality controllers, traffic, etc.) and other open source sources, such as satellite data from the European Copernicus network.

In addition, to support data collection activities foreseen in the project, a reference architecture will be developed for the platform of peripheral sensor management and for the transmission of the signals (edge gateway). A context broker will provide mechanisms for spatial aggregation and decoupling between the data generation and use, through the implementation of a Publish/Subscribe system. An open standard API will provide the components needed to ensure the portability and interoperability of open data solutions for smart cities (FIWARE NGSI API3).

All project data will be released as open data after proper anonymization, which may include an aggregate data release. The project geographical scope will be the metropolitan area of Cagliari and the lambda architecture will have the main processing node at the CRS4 headquarters in Pula (Cagliari) and a secondary node at the CRS4 site in Pirri-Cagliari.