Data Acquisition

The Data Acquisition Team are integral to Skyscanner, ensuring that our price data is relevant, accurate, and available. We also maintain and exploit the internal logging system that tracks all our sales activity, providing critical information and key performance indicators to the company.

The team's main focus is gathering of flight, price and availability data from third party websites and selected partners. Every day is different, with unique challenges ranging from implementing data collection from new websites, fixing and maintaining our links to existing partners, or setting up tracking essential to our business model.

The code-base for the web scraping library is written in Python, and XML/Regular expressions are used to parse out relevant data from third party websites and APIs. The Data & Reporting team also manage and maintain our relationships with the Global Distribution Systems (GDS) that provide pricing information for many of our affiliated airlines.

The reporting element of the team focuses on the storage of data in our internal logging system. The logging system uses a combination of Python and RabbitMQ to validate, correlate and insert the data into the high volume OLTP system based on a SQL Server platform.

This data is then exploited to provide insights and information across the company, including monitoring operational environments, dealing with commercial queries, and undertaking research projects when required. Other important responsibilities include the maintenance of the business intelligence infrastructure and governance of all data that enters the logging system.

 

Latest Jobs

Want more? Try these:

Enjoyed this article? Get more content like this direct to your inbox. Sign up for our FREE weekly newsletter for travel deals, tips, and inspiration! *