It is sad to see many financial companies burn millions of dollars on Big Data solutions and use them for silly things like keeping tap on system logs. So I was wondering where can Big Data can add value for a financial company. To start with do financial companies have significant amount of unstructured data at all? Structured data forms a big portion of data in finance unlike other industries. Financial information about various instruments are in neat tables. Price history is in time series i.e. arrays. Client data and transactions are in RDBMS. There is some need for using unstructured data though. Like for example:
- Companies have different formats to present their financial reports. This is classic case for unstructured data with text and numbers. In fact investors, banks and regulators can use advanced text analytics tools to automate conversion of unstructured data to a structured format.
- News flow effecting financial world comes in a continuous stream. This again is a candidate for unstructured data, real time processing and text analytics. Some people consider this part of Big Data. But the role of Big Data is quite limited I believe.
- Textual data like customer complaints, social media interaction etc. are strictly not financial data but do form part of data relevant for a financial company like any other company.
So the summary is I found some use for unstructured data analysis and text analytics. But the utility of a distributed file system based DB like Hadoop in a financial company still alludes me.
Follow
