This component is responsible for the complete data import process including the crawling and scraping of web-content from various sources and the import of client internal data. The received data is forwarded using the REST based communication to the respective service component for further processing: data analytics service and aggregation service.
The receiving of provided internal client data – for further forwarding and processing via the aggregation component – is also handled by this component. As it is assumed that clients (especially within the financial service industry) will be rather reluctant in having their internal data un-anonymised stored within a cloud environment client data will first be anonymized before sent to the Dydon service. For the free collection of market data coming from other sources than company internal – e.g. via the intelligent web screening and data loading –, it is assumed no anonymization is needed.