Pre-Processing Big Data: Techniques to improve quality of big data analysis

Implementing AI applications based on machine learning is a significant topic for organizations embracing digital transformation. By 2020, 30% of CIOs will include AI in their top five investment priorities according to Gartner’s Top 10 Strategic Technology Trends for 2018: Intelligent Apps and Analytics. But to deliver on the AI promise, organizations need to generate good quality data to train the algorithms. Failure to do so will result in the following scenario: "When you automate a mess, you get an automated mess."

This webinar covers:

- An introduction to machine learning use cases and challenges provided by Kirk Borne, Principal Data Scientist at Booz Allen Hamilton and top data science and big data influencer.
- How to achieve good data quality based on harmonized semantic metadata presented by Andreas Blumauer, CEO and co-founder of Semantic Web Company and a pioneer in the application of semantic web standards for enterprise data integration.
- How to apply a combined approach when semantic knowledge models and machine learning build the basis of your cognitive computing. (See Attachment: The Knowledge Graph as the Default Data Model for Machine Learning)
- Why a combination of machine and human computation approaches is required, not only from an ethical but also from a technical perspective.

How are financial service firms around the world using machine learning systems today to identify and address risk in transactional datasets?

This webinar will look at a new approach to transaction analysis and illustrate how the combination of traditional rules-based approaches can be augmented with next-generation machine learning systems to uncover more in the data, faster and more efficiently.

We will span the various applications in banking, payments, trading, and compliance; looking at a variety of use cases from bank branch transaction analysis to trading data validation.

Anyone interested in financial technology, next-generation machine learning systems and the future of the financial services industry will find this webinar of specific interest.

About the speaker:
Erik McBain, CFA is a Strategic Account Manager for MindBridge Ai, where he specializes in the deployment of emerging technologies such as artificial intelligence and machine learning systems in global financial institutions and corporations. Over his 10-year career in banking and financial services(Deutsche Bank, CIBCWM, Central Banking), Erik has been immersed in the trading, analysis, and sale of financial instruments and the deployment of new payment, banking and intelligent technologies. Erik's focus is identifying the various opportunities created through technological disruption, creating partnerships, and applying a client-centered innovation process to create transformative experiences, products, and services for his clients.

Artificial Intelligence has a huge role to play in banking, no more so than in sustainable finance. However, data is very patchy and much source data is not available to inform Sustainable Finance. The challenge as we set off on this new journey is to make sure that the data and algorithms used are transparent and unbiased.

In this session, Richard Peers, Director of Financial Services industry at Microsoft will share how disruption and new entrants are bringing new business models and technology to play in banking as in other industries like the Auto Industry

One new area is sustainable Finance, a voluntary initiative as part of the COP agreement on climate change but the data to inform the markets is a challenge. Big Data, Machine Learning and AI can help resolve this.

But with such important issues at stake, this session will outline how AI much be designed to ethical principles

Tune in to this session for a high-level view of some key trends and technologies in banking. Get insight into sustainable finance; why AI can help and why Ethical AI is important; and the Microsoft principles for Ethical AI.

We are seeing a sea change in networking. SDN has enabled improvements in network telemetry and analytics.

In this presentation, I will talk about the current challenges that are out there and how the technology change is helping us to improve the overall network telemetry. Furthermore, I will share how deep learning techniques are being used in this field. Please join this webinar to understand how the field of network telemetry is changing.

The tragedy of the commons, first described by biologist Garrett Hardin in 1968, describes how shared resources are overused and eventually depleted. He compared shared resources to a common grazing pasture; in this scenario, everyone with rights to the pasture acting in self-interest for the greatest short-term personal gain depletes the resource until it is no longer viable.

The banking ecosystem and the data that binds it together is not all that different. For many years, through miss-selling scandals, cookie cutter products and dumb mass-marketing have seen players acting in their own interest in accordance to what they believe the ecosystem should look like, how it should evolve and who controls it.

But with the introduction of open banking, there are signs that new banking ecosystems are set to thrive. Taking Hardin’s notion, collaboration in the open banking future could benefit everyone in the ecosystem – the traditional banks, the FinTechs, the tech titans with their expertise in delivering services at scale, and yet-to-be-defined participants, likely to include the large data players such as energy firms, retailers and telcos.

Join me to explore the Open Future.

This talk tells the story of implementation and optimization of a sparse logistic regression algorithm in spark. I would like to share the lessons I learned and the steps I had to take to improve the speed of execution and convergence of my initial naive implementation. The message isn’t to convince the audience that logistic regression is great and my implementation is awesome, rather it will give details about how it works under the hood, and general tips for implementing an iterative parallel machine learning algorithm in spark.

The talk is structured as a sequence of “lessons learned” that are shown in form of code examples building on the initial naive implementation. The performance impact of each “lesson” on execution time and speed of convergence is measured on benchmark datasets.

You will see how to formulate logistic regression in a parallel setting, how to avoid data shuffles, when to use a custom partitioner, how to use the ‘aggregate’ and ‘treeAggregate’ functions, how momentum can accelerate the convergence of gradient descent, and much more. I will assume basic understanding of machine learning and some prior knowledge of spark. The code examples are written in scala, and the code will be made available for each step in the walkthrough.

Lorand is a data scientist working on risk management and fraud prevention for the payment processing system of Zalando, the leading fashion platform in Europe. Previously, Lorand has developed highly scalable low-latency machine learning algorithms for real-time bidding in online advertising.

You can do a lot with a Raspberry and ASF projects. From a tiny object
connected to the internet to a small server application. The presentation
will explain and demo the following:

- Raspberry as small server and captive portal using httpd/tomcat.
- Raspberry as a IoT Sensor collecting data and sending it to ActiveMQ.
- Raspberry as a Modbus supervisor controlling an Industruino
(Industrial Arduino) and connected to ActiveMQ.

The 10x growth of transaction volumes, 50x growth in data volumes and drive for real-time visibility and responsiveness over the last decade have pushed traditional technologies including databases beyond their limits. Your choices are either buy expensive hardware to accelerate the wrong architecture, or do what other companies have started to do and invest in technologies being used for modern hybrid transactional analytical applications (HTAP).

Learn some of the current best practices in building HTAP applications, and the differences between two of the more common technologies companies use: Apache® Cassandra™ and Apache® Ignite™. This session will cover:

- The requirements for real-time, high volume HTAP applications
- Architectural best practices, including how in-memory computing fits in and has eliminated tradeoffs between consistency, speed and scale
- A detailed comparison of Apache Ignite and GridGain® for HTAP applications

About the speaker: Denis Magda is the Director of Product Management at GridGain Systems, and Vice President of the Apache Ignite PMC. He is an expert in distributed systems and platforms who actively contributes to Apache Ignite and helps companies and individuals deploy it for mission-critical applications. You can be sure to come across Denis at conferences, workshop and other events sharing his knowledge about use case, best practices, and implementation tips and tricks on how to build efficient applications with in-memory data grids, distributed databases and in-memory computing platforms including Apache Ignite and GridGain.

Before joining GridGain and becoming a part of Apache Ignite community, Denis worked for Oracle where he led the Java ME Embedded Porting Team -- helping bring Java to IoT.

Attend this session to learn how to easily share state in-memory across multiple Spark jobs, either within the same application or between different Spark applications using an implementation of the Spark RDD abstraction provided in Apache Ignite. During the talk, attendees will learn in detail how IgniteRDD – an implementation of native Spark RDD and DataFrame APIs – shares the state of the RDD across other Spark jobs, applications and workers. Examples will show how IgniteRDD, with its advanced in-memory indexing capabilities, allows execution of SQL queries many times faster than native Spark RDDs or Data Frames.

Akmal Chaudhri has over 25 years experience in IT and has previously held roles as a developer, consultant, product strategist and technical trainer. He has worked for several blue-chip companies such as Reuters and IBM, and also the Big Data startups Hortonworks (Hadoop) and DataStax (Cassandra NoSQL Database). He holds a BSc (1st Class Hons.) in Computing and Information Systems, MSc in Business Systems Analysis and Design and a PhD in Computer Science. He is a Member of the British Computer Society (MBCS) and a Chartered IT Professional (CITP).

When monitoring an increasing number of machines, the infrastructure and tools need to be rethinked. A new tool, ExDeMon, for detecting anomalies and raising actions, has been developed to perform well on this growing infrastructure. Considerations of the development and implementation will be shared.

Daniel has been working at CERN for more than 3 years as Big Data developer, he has been implementing different tools for monitoring the computing infrastructure in the organisation.

As data analytics becomes more embedded within organizations, as an enterprise business practice, the methods and principles of agile processes must also be employed.

Agile includes DataOps, which refers to the tight coupling of data science model-building and model deployment. Agile can also refer to the rapid integration of new data sets into your big data environment for "zero-day" discovery, insights, and actionable intelligence.

The Data Lake is an advantageous approach to implementing an agile data environment, primarily because of its focus on "schema-on-read", thereby skipping the laborious, time-consuming, and fragile process of database modeling, refactoring, and re-indexing every time a new data set is ingested.

Another huge advantage of the data lake approach is the ability to annotate data sets and data granules with intelligent, searchable, reusable, flexible, user-generated, semantic, and contextual metatags. This tag layer makes your data "smart" -- and that makes your agile big data environment smart also!

With new technologies such as Hive LLAP or Spark SQL, do you still need a data warehouse or can you just put everything in a data lake and report off of that? No! In the presentation, James will discuss why you still need a relational data warehouse and how to use a data lake and an RDBMS data warehouse to get the best of both worlds.

James will go into detail on the characteristics of a data lake and its benefits and why you still need data governance tasks in a data lake. He'll also discuss using Hadoop as the data lake, data virtualization, and the need for OLAP in a big data solution, and he will put it all together by showing common big data architectures.

Watson is a computer system capable of answering questions posed in natural language. Watson was named after IBM's first CEO, Thomas J. Watson. The computer system was specifically developed to answer questions on the quiz show Jeopardy! (where it beat its human competitors) and was then used in commercial applications, the first of which was helping with lung cancer treatment.

NetApp is now using IBM Watson in Elio, a virtual support assistant that responds to queries in natural language. Elio is built using Watson’s cognitive computing capabilities. These enable Elio to analyze unstructured data by using natural language processing to understand grammar and context, understand complex questions, and evaluate all possible meanings to determine what is being asked. Elio then reasons and identifies the best answers to questions with help from experts who monitor the quality of answers and continue to train Elio on more subjects.

Elio and Watson represent an innovative and novel use of large quantities of unstructured data to help solve problems, on average, four times faster than traditional methods. Join us at this webcast, where we’ll discuss:

•The challenges of utilizing large quantities of valuable yet unstructured data
•How Watson and Elio continuously learn as more data arrives, and navigates an ever growing volume of technical information
•How Watson understands customer language and provides understandable responses

Learn how these new and exciting technologies are changing the way we look at and interact with large volumes of traditionally hard-to-analyze data.

After the webcast, check-out the Q&A blog http://www.sniacloud.com/?p=296

It can be hard to keep up with the rapidly changing BI landscape. But it doesn't have to be. Reserve your spot at Qlik's annual BI Trends Webinar.

In this global webinar live replay, we’ll reveal the top BI Trends for the coming year and how they can help you transform your data. Join Qlik’s Global Market Intelligence lead and former Gartner analyst Dan Sommer to learn why 2018 is the year for the “desilofication of data.”

Recent events like the Equifax data leak and new regulations like the EU's General Data Protection Regulation have increased the urgency for further change in the BI landscape and to move data out of silos.

What is the right strategy and framework?
How can you easily move from "all data," to "combinations of data," to "data insights"?
Can data literacy and augmented intelligence create a data-driven culture?
The volume of data available to decision makers continues to be massive, and is growing faster than our ability to consume it. Learn how to move your data out of silos and turn your data into insights.

RIDE is an all-in-one, multi-user, multi-tenant, secure and scalable platform for developing and sharing Data Science and Analytics, Machine Learning (ML) and Artificial Intelligence (AI) solutions in R, Python and SQL.

RIDE supports developing in notebooks, editor, RMarkdown, shiny app, Bokeh and other frameworks. Supported by R-Brain’s optimized kernels, R and Python 3 have full language support, IntelliSense, debugger and data view. Autocomplete and content assistant are available for SQL and Python 2 kernels. Spark (standalone) and Tesnsorflow images are also provided.

Using Docker in managing workspaces, this platform provides an enhanced secure and stable development environment for users with a powerful admin control for controlling resources and level of access including memory usage, CPU usage, and Idle time.

The latest stable version of IDE is always available for all users without any need of upgrading or additional DevOps work. R-Brain also delivers customized development environment for organizations who are able to set up their own Docker registry to use their customized images.

The RIDE Platform is a turnkey solution that increases efficiency in your data science projects by enabling data science teams to work collaboratively without a need to switch between tools. Explore and visualize data, share analyses, all in one IDE with root access, connection to git repositories and databases.

IT is a key player in the digital and cognitive transformation of business processes delivering solutions for improved business value with analytics. This session will step by step explain the journey to secure production while adopting new analytics technologies leveraging mainframe core business assets

Big Data, Artificial Intelligence and Machine Learning

We will discuss how Big Data, Artificial Intelligence and Machine learning are rapidly impacting businesses and customers, enabling another massive shift through technology enablement. Todd DeCapua will share how these capabilities are being leveraged in Performance Engineering now, and into the future.

Join us for the next Quality & Testing SIG Talk on Tuesday, January 9, 2018: http://www.vivit-worldwide.org/events/EventDetails.aspx?id=1041157&group=.

The newest buzzword after Big Data is AI. From Google search to Facebook messenger bots, AI is also everywhere.

Machine learning has gone mainstream. Organizations are trying to build competitive advantage with AI and Big Data.

But, what does it take to build Machine Learning applications? Beyond the unicorn data scientists and PhDs, how do you build on your big data architecture and apply Machine Learning to what you do?

This talk will discuss technical options to implement machine learning on big data architectures and how to move forward.

Researchers generate huge amounts of valuable unstructured data and articles from research every day. The potential for this information is huge: cancer and pharmaceutical breakthroughs, advances in technology and cultural research that can improve the world we live in.

This webinar discusses how text mining and Machine Learning can be used to make connections across this broad range of files and help drive innovation and research. We discuss using Kubernetes microservices to analyse the data and then applying Machine Learning and graph databases to simplify the reuse of the data.