Hunting Criminals with Hybrid Analytics, Semi-supervised Learning, & Feedback

Presented by

David Talby, CTO, Pacific AI

About this talk

Fraud detection is a classic adversarial analytics challenge: As soon as an automated system successfully learns to stop one scheme, fraudsters move on to attack another way. Each scheme requires looking for different signals (i.e. features) to catch; is relatively rare (one in millions for finance or e-commerce); and may take months to investigate a single case (in healthcare or tax, for example) – making quality training data scarce. This talk will cover a code walk-through, the key lessons learned while building such real-world software systems over the past few years. We'll look for fraud signals in public email datasets, using IPython and popular open-source libraries (scikit-learn, statsmodel, nltk, etc.) for data science and Apache Spark as the compute engine for scalable parallel processing. David will iteratively build a machine-learned hybrid model – combining features from different data sources and algorithmic approaches, to catch diverse aspects of suspect behavior: - Natural language processing: finding keywords in relevant context within unstructured text - Statistical NLP: sentiment analysis via supervised machine learning - Time series analysis: understanding daily/weekly cycles and changes in habitual behavior - Graph analysis: finding actions outside the usual or expected network of people - Heuristic rules: finding suspect actions based on past schemes or external datasets - Topic modeling: highlighting use of keywords outside an expected context - Anomaly detection: Fully unsupervised ranking of unusual behavior Apache Spark is used to run these models at scale – in batch mode for model training and with Spark Streaming for production use. We’ll discuss the data model, computation, and feedback workflows, as well as some tools and libraries built on top of the open-source components to enable faster experimentation, optimization, and productization of the models.

Related topics:

More from this channel

Upcoming talks (13)
On-demand talks (3529)
Subscribers (181981)
This channel features presentations by leading experts in the field of information security. From application, computer, network and Internet security to access control management, data privacy and other hot topics, you will walk away with practical advice for your strategic and tactical information security initiatives.