Data Loading

This component is responsible for the complete data import process including the crawling and scraping of web-content from various sources and the import of client internal data. The received data is forwarded using the REST based communi­cation to the respective service component for further processing: data analytics service and aggregation service.

The receiving of provided internal client data – for further forwarding and process­ing via the aggregation component – is also handled by this component. As it is assum­ed that clients (especially within the financial service industry) will be rather reluctant in having their internal data un-anonymised stored within a cloud environ­ment client data will first be anonymized before sent to the Dydon service. For the free collection of market data coming from other sources than company internal – e.g. via the intelligent web screening and data loading –, it is assumed no anonymization is needed.