PorphyrioIntelligent data management
software for the poultry industry

Explore the case

About the client

Porphyrio is a reliable data partner for the poultry industry. By developing cloud-based IoT and big data solutions, Porphyrio enables each customer to optimize their business, online and in real-time. As such, their customers can take better strategic decisions.


The problem

A massive amount of both manual and automatic data is entering the Porphyrio system each day. The data describes the daily processes in the poultry industry. It contains information about the environment, such as temperature, humidity, and numerous other gas densities. But also the number of eggs, the number of newborn poultry en the number of deaths are gathered by the system with a fixed frequency.

However, the data may contain noise and other artifacts causing systems to crash or to output strange results. Detecting these errors and preventing them from happening in the future is essential in delivering a reliable and accurate platform to the support managers.


The solution

Together with Porphyrio, Sentigrate developed a data pipeline to reliably and accurately ingest all sorts of data into their system.

A combination of Nifi and Airflow allowed us to programmatically define the data flow and corresponding steps in Python. This way the data can be transformed, manipulated and cleaned in a testable and extensible way, allowing for a clear separation of concern for each step.

The results of each intermediate step were saved to a separate table in a Postgres database, which ensured easier debugging but also allowed us to easily resume the data flow when necessary.




Increase in automation

The developed data pipeline requires no manual interactions and runs fully automated. But even when something goes wrong, the tools and self-developed visualizations allow full insight into the process and identify the exact point of failure. Using this setup, Porphyrio is prepared to scale their data ingestion without losing reliability.

Porphyrio explanatory video

Company Facts

Number and facts about the case