Primitives for AI

Let’s say I want to build a financial market prediction model based on news flow. The current way to do this is:

  • Collect lot of news articles
  • Collect financial market movement data
  • Use machine learning models (black-box or otherwise) to predict markets from news

This is not a scalable model of operation for building AI/ML solutions.

Continue reading

Trending data-driven consulting

‘In God we trust, all other must bring data’ said late William Edward Deming – the famous  American statistician and professor. Gut based decision making is out, subjectivity is out, generic experience based decisions are out – Data driven decisions are in.… Continue reading

Insightful Decision-making

 

Many businesses use data insights to support their decisions instead of driving their actions. After all, data is only valuable if you can translate it into actionable insights. Your organization might be sitting on the world’s largest data pile, but it is useless unless you have the means to translate it into insights that drives your business.… Continue reading

Out-of-core Analytics using MonetDB

AAEAAQAAAAAAAAyBAAAAJGI5MTQyZmJkLTVlNmYtNDYxMy1hYTU4LTliOTQ4MjAyYWEzOQ

Most often than not, the amount of data one needs to analyse will fit in a hard disc. It may not fit in memory though. That’s where databases trump over in-memory solutions like R or Pandas (Python) for analytics. For sometime we have been arguing that column stores/columnar databases will be ideal for analytics (https://www.linkedin.com/pulse/column-stores-future-analytics-gopi-suvanam).… Continue reading

Columnar DBs: Future of Structured Data

Library_of_Congress_Entrance_Hall_Columns

Contrary to popular perception, structured data analytics still forms a big potion of overall analytics/big data activity. Although unstructured data (text, data with variety etc.) is gaining more and more importance, structure data (transactions, numeric data etc.) still gives a lot of actionable insights for businesses.… Continue reading

Machine learning: what is and what is not

images.duckduckgo.com

As elaborated earlier, Artificial Intelligence and machine learning are two different things. For a 19th century man even a simple calculating machine could constitute AI. But we all now agree that even more advanced computer algorithms are not machine learning but human intelligence codified into machine.

Continue reading

Not So Big Data!!!

logo-SoBigData-DEFINITIVO

Imagine you are an organization sitting on a tons of transactional and other business data which is stored in structured database. Now, you have to analyse it on a daily basis to get vital insights about sales, customers, channels and many other business parameters, what will be your approach?… Continue reading

Why machine learning is worth the hype

Machine learning has seen significant hype in last one year. Applications range from chatbots to image searches to speech recognition to financial market prediction and more! Well the hype is all worth it. (Except may be for financial market prediction. Not may be, definitely.) ML will live upto its hype as:

It reduces friction in customer interactions.… Continue reading

Story of Artificial Intelligence

The history of Artificial Intelligence is as fascinating as the future it promises. For many, AI is a recent phenomenon, rising with the rise of connected world. But research and speculation in AI have been much older than most believe. Automaton have been described since antiquity.… Continue reading

Column stores: The future of analytics

The most common myth pertain to BigData is considering BigData and Hadoop as synonymous. Data in most business cases is structured and small enough to fit in a hard disc. In very extreme cases (think FB/Google level) is the data larger than something that cannot fit into a hard-disc.… Continue reading

Death of OLAP cubes

At one point analytics meant MIS and MIS meant OLAP Cubes. For the uninitiated, OLAP is online analytics processing and OLAP cubes are aggregated cuts of various business KPIs across different time and business factor dimensions. A simple example could be sales across different products, geographies and different months.… Continue reading

Functional Thinking for Analytics and Machine Learning

Object oriented (OO) has dominated programming design patterns for quite sometime now. In the initial stages of web development and mobile apps, the main requirement was to extract, store and present data. OO is considered ideal for this. AS move away from basic data representation and expect more from computing i.e.… Continue reading