Friday, November 24, 2017

Free Deep Learning Book (MIT Press)

The Deep Learning textbook is a resource intended to help students and practitioners enter the field of machine learning in general and deep learning in particular. The online version of the book is now complete and will remain available online for free.
Source for picture: click here
For more information, click here. The book is also available on Amazon, and also here (MIT Press). 


We plan to offer lecture slides accompanying all chapters of this book. We currently offer slides for only some chapters. If you are a course instructor and have your own lecture slides that are relevant, feel free to contact us if you would like to have your slides linked or mirrored from this site.
  1. Introduction
    • Presentation of Chapter 1, based on figures from the book [.key] [.pdf]
    • Video of lecture by Ian and discussion of Chapter 1 at a reading group in San Francisco organized by Alena Kruchkova
  2. Linear Algebra [.key][.pdf]
  3. Probability and Information Theory [.key][.pdf]
  4. Numerical Computation [.key] [.pdf] [youtube]
  5. Machine Learning Basics [.key] [.pdf]
  6. Deep Feedforward Networks [.key] [.pdf]
    • Video (.flv) of a presentation by Ian and a group discussion at a reading group at Google organized by Chintan Kaur.
  7. Regularization for Deep Learning [.pdf] [.key]
  8. Optimization for Training Deep Models
    • Gradient Descent and Structure of Neural Network Cost Functions [.key] [.pdf]
      These slides describe how gradient descent behaves on different kinds of cost function surfaces. Intuition for the structure of the cost function can be built by examining a second-order Taylor series approximation of the cost function. This quadratic function can give rise to issues such as poor conditioning and saddle points. Visualization of neural network cost functions shows how these and some other geometric features of neural network cost functions affect the performance of gradient descent.
    • Tutorial on Optimization for Deep Networks [.key] [.pdf]
      Ian's presentation at the 2016 Re-Work Deep Learning Summit. Covers Google Brain research on optimization, including visualization of neural network cost functions, Net2Net, and batch normalization.
    • Batch Normalization [.key] [.pdf]
    • Video of lecture / discussion: This video covers a presentation by Ian and group discussion on the end of Chapter 8 and entirety of Chapter 9 at a reading group in San Francisco organized by Taro-Shigenori Chiba.
  9. Convolutional Networks
    • Convolutional Networks [.key][.pdf]
      A presentation summarizing Chapter 9, based directly on the textbook itself.
    • Video of lecture / discussion: This video covers a presentation by Ian and group discussion on the end of Chapter 8 and entirety of Chapter 9 at a reading group in San Francisco organized by Taro-Shigenori Chiba.
  10. Sequence Modeling: Recurrent and Recursive Networks
    • Sequence Modeling [.pdf] [.key]
      A presentation summarizing Chapter 10, based directly on the textbook itself.
    • Video of lecture / discussion. This video covers a presentation by Ian and a group discussion of Chapter 10 at a reading group in San Francisco organized by Alena Kruchkova.
  11. Practical Methodology [.key][.pdf] [youtube]
  12. Applications [.key][.pdf]
  13. Linear Factors [.key][.pdf]
  14. Autoencoders [.key][.pdf]
  15. Representation Learning [.key][.pdf]
  16. Structured Probabilistic Models for Deep Learning[.key][.pdf]

Friday, November 17, 2017

Explainable Artificial Intelligence (XAI)

Dramatic success in machine learning has led to a torrent of Artificial Intelligence (AI) applications. Continued advances promise to produce autonomous systems that will perceive, learn, decide, and act on their own. However, the effectiveness of these systems is limited by the machine’s current inability to explain their decisions and actions to human users. The Department of Defense is facing challenges that demand more intelligent, autonomous, and symbiotic systems. Explainable AI—especially explainable machine learning—will be essential if future warfighters are to understand, appropriately trust, and effectively manage an emerging generation of artificially intelligent machine partners.
The Explainable AI (XAI) program aims to create a suite of machine learning techniques that:
  • Produce more explainable models, while maintaining a high level of learning performance (prediction accuracy); and
  • Enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners.
New machine-learning systems will have the ability to explain their rationale, characterize their strengths and weaknesses, and convey an understanding of how they will behave in the future. The strategy for achieving that goal is to develop new or modified machine-learning techniques that will produce more explainable models. These models will be combined with state-of-the-art human-computer interface techniques capable of translating models into understandable and useful explanation dialogues for the end user. Our strategy is to pursue a variety of techniques in order to generate a portfolio of methods that will provide future developers with a range of design options covering the performance-versus-explainability trade space.
Figure 1: XAI Concept
The XAI program will focus the development of multiple systems on addressing challenges problems in two areas: (1) machine learning problems to classify events of interest in heterogeneous, multimedia data; and (2) machine learning problems to construct decision policies for an autonomous system to perform a variety of simulated missions. These two challenge problem areas were chosen to represent the intersection of two important machine learning approaches (classification and reinforcement learning) and two important operational problem areas for the Department of Defense (intelligence analysis and autonomous systems).
XAI research prototypes will be tested and continually evaluated throughout the course of the program. At the end of the program, the final delivery will be a toolkit library consisting of machine learning and human-computer interface software modules that could be used to develop future explainable AI systems. After the program is complete, these toolkits would be available for further refinement and transition into defense or commercial applications.

Tuesday, November 14, 2017

Demystifying AI, Machine Learning, and Deep Learning

Learn about AI, machine learning, supervised learning, unsupervised learning, classification, decision trees, clustering, deep learning, and algorithms.

Deep learning, machine learning, artificial intelligence — all buzzwords that represent the future of analytics. In this post, we will explain what machine learning and deep learning are at a high level with some real-world examples. In future posts, we will explore vertical use cases. The goal of this is not to turn you into a data scientist but to give you a better understanding of what you can do with machine learning. Machine learning is becoming more accessible to developers, and data scientists work with domain experts, architects, developers, and data engineers, so it is important for everyone to have a good understanding of the possibilities. Every piece of information that your business generates has the potential to add value. This post and future posts are meant to provoke a review of your own data to identify new opportunities.
ML examples

What Is Artificial Intelligence?

Throughout the history of AI, the definition has been continuously redefined. AI is an umbrella term (the idea started in the 50s); machine learning is a subset of AI and deep learning is a subset of ML.
In 1985, when I was a student interning at the NSA, AI was also a very hot topic. At the NSA, I even took an MIT video (VCR) class on AI about expert systems. Expert systems capture an expert's knowledge in a rules engine. Rules engines have a wide use in industries such as finance and healthcare, and more recently for event processing, but when data is changing, rules can become difficult to update and maintain. Machine learning has the advantage that it learns from the data, and it can provide data-driven probabilistic predictions.
According to Ted Dunning, it is better to use precise terminology like machine learning ordeep learning instead of the term "AI" because before we get something to work well, we call it AI; afterward, we always call it something else. AI is better used as a word for the next frontier.

How Has Analytics Changed in the Last 10 Years?

According to Thomas Davenport in the HBR, analytical technology has changed dramatically over the last decade, with more powerful and less expensive distributed computing across commodity servers, streaming analytics, and improved machine learning technologies, enabling companies to store and analyze both far more data and many different types of it.
Traditionally, data was stored on a RAID system, sent to a multi-core server for processing, and sent back for storage, which caused a bottleneck on data transfer and was expensive. With file and table storage like MapR-XD and MapR-DB, data is distributed across a cluster and Hadoop technologies like MapReduce, Pig, and Hive send the computing task to where the data resides.
Technologies like Apache Spark speed up parallel processing of distributed data even more with iterative algorithms by caching data in-memory across iterations and using lighter weight threads.
Apache Spark
MapR Event Streams, a new distributed messaging system for streaming event data at scale, combined with Stream processing like Apache Spark streaming or Apache Flink speed up parallel processing of real-time events with machine learning models.
MapR Streams
Graphical Processing Units (GPUs) have sped up multi-core servers for parallel processing. A GPU has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously, whereas a CPU consists of a few cores optimized for sequential serial processing. In terms of potential performance, the evolution from the Cray-1 to today’s clusters with lots of GPU’s is roughly a million times what was once the fastest computer on the planet at a tiny fraction of the cost.

What Is Machine Learning?

Machine learning uses algorithms to find patterns in data and then uses a model that recognizes those patterns to make predictions on new data.
In general, machine learning may be broken down into types: supervised, unsupervised, and in between those two. Supervised learning algorithms use labeled data and unsupervised learning algorithms find patterns in unlabeled data. Semi-supervised learning uses a mixture of labeled and unlabeled data. Reinforcement learning trains algorithms to maximize rewards based on feedback.

Supervised Learning

Supervised algorithms use labeled data in which both the input and target outcome, or label, are provided to the algorithm.
Supervised Learning
Supervised learning is also called predictive modeling or predictive analytics because you build a model that is capable of making predictions. Some examples of predictive modeling are classification and regression. Classification identifies which category an item belongs to (for example whether a transaction is fraud or not fraud) based on labeled examples of known items (for example, transactions known to be fraud or not). Logistic regression predicts a probability — for example, the probability of fraud. Linear regression predicts a numeric value — for example, the amount of fraud.
Some examples of classification include:
  • Credit card fraud detection (fraud, not fraud).
  • Credit card application (good credit, bad credit).
  • Email spam detection (spam, not spam).
  • Text sentiment analysis (happy, not happy).
  • Predicting patient risk (high-risk patient, low-risk patient).
  • Classifying a tumor as malignant or not.
Some examples of logistic regression (or other algorithms) include:
  • Given historical car insurance fraudulent claims and features of the claims such as the age of the claimant, the claimed amount, and the severity of the accident, predict the probability of fraud.
  • Given patient characteristics, predict the probability of congestive heart failure.
Some examples of linear regression include:
  • Given historical car insurance fraudulent claims and features of the claims such as the age of the claimant, the claimed amount, and the severity of the accident, predict the amount of fraud.
  • Given historical real estate sales prices and features of houses (i.e. square feet, number of bedrooms, location), predict a house’s price.
  • Given historical neighborhood crime statistics, predict crime rate.
There are other supervised and unsupervised learning algorithms shown below, which we won’t go over, but we will look at one example of each in more detail.

Classification Example: Debit Card Fraud

Classification takes a set of data with known labels and pre-determined features and learns how to label new records based on that information. Features are the “if” questions that you ask. The label is the answer to those questions.
Let’s go through an example of debit card fraud.
  • What are we trying to predict?
    • Whether a debit card transaction is fraud.
    • Fraud is the label (true or false).
  • What are the “if” questions or properties that you can use to make predictions?
    • Is the amount spent today > historical average?
    • Are there transactions in multiple countries today?
    • Are the number of transactions today > historical average?
    • Are the number of new merchant types today high compared to the last three months?
    • Are there multiple purchases today from merchants with a category code of risk?
    • Is there unusual signing activity today compared to using historically using PIN?
    • Are there new state purchases compared to the last three months?
    • Are there foreign purchases today compared to the last three months?
To build a classifier model, you extract the features of interest that most contribute to the classification.

Decision Trees

Decision trees create a model that predicts the class or label based on several input features. Decision trees work by evaluating a question containing a feature at every node and selecting a branch to the next node based on the answer. A possible decision tree for predicting debit card fraud is shown below. The feature questions are the nodes, and the answers “yes” or “no” are the branches in the tree to the child nodes. (Note that a real tree would have more nodes.)
  • Q1: Is the amount spent in 24 hours > average?
    • Yes
  • Q2: Are there multiple purchases today from risky merchants?
    • Yes fraud = 90%
    • Not fraud = 50%
Decision Tree
Decision trees are popular because they are easy to visualize and explain. The accuracy of models can be improved by combining algorithms with ensemble methods. An ensemble example is a random forest, which combines multiple random subsets of decision trees.

Unsupervised Learning

Unsupervised learning, also sometimes called descriptive analytics, does not have labeled data provided in advance. These algorithms discover similarities or regularities in the input data. An example of unsupervised learning is grouping similar customers based on purchase data.
Unsupervised Learning


In clustering, an algorithm classifies inputs into categories by analyzing similarities between input examples. Some clustering use cases include:
  • Search results grouping.
  • Grouping similar customers.
  • Grouping similar patients.
  • Text categorization.
  • Network security anomaly detection (finds what is not similar, the outliers from clusters).
The K-means algorithm groups observations into K clusters in which each observation belongs to the cluster with the nearest mean from its cluster center.
An example of clustering is a company that wants to segment its customers in order to better tailor products and offerings. Customers could be grouped on features such as demographics and purchase histories. Clustering with unsupervised learning is often combined with supervised learning in order to get more valuable results. For example, in this banking customer 360 use case, customers were first segmented based on answers to a survey. The customer groups were analyzed and labeled with customer personas. These labels were then linked by customer ID with features such as types of accounts and purchases. Finally, supervised machine learning was applied and tested with the labeled customers, allowing to link the survey customer personas with their banking actions and provide insights.

Deep Learning

Deep learning is the name for multilayered neural networks, which are networks composed of several “hidden layers” of nodes between the input and output. There are many variations of neural networks, which you can learn more about on this neural network cheat sheet. Improved algorithms, GPUs, and massively parallel processing (MPP) have given rise to networks with thousands of layers. Each node takes input data and a weight and outputs a confidence score to the nodes in the next layer until the output layer is reached where the error of the score is calculated. With backpropagation inside of a process called gradient descent, the errors are sent back through the network again and the weights are adjusted improving the model. This process is repeated thousands of times, adjusting a model’s weights in response to the error it produces until the error can’t be reduced anymore.
During this process, the layers learn the optimal features for the model, which has the advantage that features do not need to be predetermined. However, this has the disadvantage that the model’s decisions are not explainable. Because explaining the decisions can be important, researchers are developing new ways to understand the black box of deep learning.
There are different variations of Deep Learning Algorithms, which can be used with the Distributed Deep Learning Quick Start Solution from MapR to build data-driven applications such as the following:
Deep Learning QSS
  • Deep neural networks for improved traditional algorithms.
    • Finance: Enhanced fraud detection through identification of more complex patterns.
    • Manufacturing: Enhanced identification of defects based on deeper anomaly detection.
  • Convolutional neural networks for images.
    • Retail: In-store activity analysis of video to measure traffic.
    • Satellite images: Labeling terrain and classifying objects.
    • Automotive: Recognition of roadways and obstacles.
    • Healthcare: Diagnostic opportunities from x-rays, scans, etc.
    • Insurance: Estimating claim severity based on photographs.
  • Recurrent neural networks for sequenced data.
    • Customer satisfaction: Transcription of voice data to text for NLP analysis.
    • Social media: Real-time translation of social and product forum posts.
    • Photo captioning: Search archives of images for new insights.
    • Finance: Predicting behavior based via time series analysis (also enhanced recommendation systems).

Tuesday, September 19, 2017

The Future of Big Data: 10 Predictions You Should Be Aware Of

Come 2020, every person in the world will be creating 7 MBs of data every second. We have already created more data in past couple of years than in the entire history of human kind. Big data has taken the world by storm and there are no signs of slowing down. You might be thinking, “Where would big data industry go from here?” Here are 10 big data predictions that will answer your intriguing question.

1. Machine Learning Will Be the Next Big Thing in Big Data

One of the hottest technology trends today is machine learning and it will play a big part in the future of big data as well. According to Ovum, Machine learning will be at the forefront of the big data revolution. It will help businesses in preparing data and conduct predictive analysis so that businesses can overcome future challenges easily.

2. Privacy Will Be the Biggest Challenge

Whether it is the internet of things or big data, the biggest challenge for emerging technologies has been security and privacy of data. The volume of data we are creating right now and the volume of data that will be created in the future will make privacy even more important as stakes will be much higher. According to Gartner, more than 50% of business ethics violation by 2018 will be data related. Data security and privacy concerns will be the biggest hurdle for big data industry and if it fails to cope with it in an effective manner, we will see a long list of technology trends that became a fad very quickly.

3. Chief Data Officer: A New Position Will Emerge

You might be familiar with Chief Executive Officer (CEO), Chief Marketing Officer (CMO) and Chief Information Officer (CIO) but have you ever heard about Chief Data Officer (CDO)? If your answer is no, do not worry because you will soon come to know about it. According to Forrester, we will see the emergence of chief data officer as the new position and businesses will appoint chief data officers. Although, the appointment of chief data officer solely depend on the type of business and its data needs but the wider adoption of big data technologies across enterprises, hiring a chief data officer will become the norm.

4. Data Scientists Will Be In High Demand

If you are still not quite sure about which career path to choose then, there is no better time to start your career in data sciences. As the volume of data grows and big data grows bigger, demand for data scientists, analysts and data management experts will shoot up.  The gap between the demand for data professionals and the availability will widen. This will help data scientists and analysts draw higher salaries. What are you waiting for? Dive into the world of data sciences and have a brighter future.

5. Businesses Will Buy Algorithms, Instead of Software

We will see a 360-degree shift in business approach towards software. More and more businesses will look to purchase algorithm instead of creating their own. After buying an algorithm, businesses can add their own data to it. It provides businesses with more customization options as compared to when they are buying software. You cannot tweak software according to your needs. In fact, it is the other way around. Your business will have to adjust according to the software processes but all this will end soon with algorithms selling services taking center stage.

6. Investments in Big Data Technologies Will Skyrocket

According to IDC analysts, “Total revenues from big data and business analytics will rise from $122 billion in 2015 to $187 billion in 2019.” Business spending on big data will surpass $57 billion dollars this year. Although, the business investments in big data might vary from industry to industry, the increase in big data spending will remain consistent overall.  Manufacturing industry will spend the most on big data technology while health care, banking, and resource industries will be the fastest to adopt.

7. More Developers Will Join the Big Data Revolution

According to statistics, there are six million developers currently working with big data and using advanced analytics. This makes up more than 33% of developers in the world. What’s even more amazing is that big data is just getting starting so will see a surge in a number of developer developing applications for big data in years to come. With the financial rewards in terms of higher salaries involved, developers will love to create applications that can play around with big data.

8. Prescriptive Analytics Will Become an Integral Part of BI Software

Gone are the days when businesses have to purchase dedicated software for everything. Today, businesses demand single software that provides all the features they need and software companies and giving them that. Business intelligence software is also following that trend and we will see prescriptive analysis capabilities added to this software in the future.
IDC predicts that half of the business analytics software will incorporate prescriptive analytics build on cognitive computing functionality. This will help businesses to make intelligent decisions at the right time. With intelligence built into the software, you can sift through large amounts of data quickly and get a competitive advantage over your competitors.

9. Big Data Will Help You Break Productivity Records

None of your future investments will deliver a higher return on your investment than if you invest in big data, especially when it comes to boosting your business productivity. To give you a better idea, let us put numbers into perspective. According to IDC, organizations that invest in this technology and attain capabilities to analyze large amounts of data quickly and extract actionable information can get an extra $430 billion in terms of productivity benefits over their competitors. Yes, you read that right, $430 billion dollars. Remember, actionable is the key word here. You need actionable information to take your productivity to new heights.

10. Big Data Will Be Replaced By Fast and Actionable Data

According to some big data experts, big data is dead. They argue that businesses do not even use a small portion of data they have access to and big does not always mean better. Sooner rather than later, big data will be replaced by fast and actionable data, which will help businesses, take the right decisions at the right time. Having tremendous amounts of data will not give you a competitive advantage over your competitors but how effectively and quickly you analyze the data and extract actionable information from it will.

Wednesday, September 6, 2017

Tài liệu hướng dẫn sử dụng Google Analytics

Đã từ lâu, Google Analytics đã đóng một vai trò quan trọng trong việc hỗ trợ quản lý các hoạt động của website và giúp các quản trị viên gia tăng hiệu quả hoạt động cho website của mình. Công cụ Analytics của Google cũng là một trong những công cụ hỗ trợ SEO rất tốt, cho phép chúng ta đo lường và gia tăng hiệu quả SEO.
Đã đến lúc chúng ta cần đưa công cụ này vào trong công việc SEO như là một trong những công cụ không thể thiếu cho tất cả các chiến dịch SEO. Để hỗ trợ bạn đọc trong việc sử dụng công cụ này một cách hiệu quả, xin gửi đến bạn đọc tài liệu hướng dẫn tổng quan về cách sử dụng Google Analytics từ cơ bản đến nâng cao của tác giả Quang Huy – Bá Cường:
(Download tài liệu này tại đây)

Sau đây là một vài các ý chính có trong tài liệu này: 
  1. Tổng quan về Google Analytics (Phần 1: Giá trị cốt lõi)
  2. Hướng dẫn sử dụng giao diện Google Analytics (Phần 2: Giao diện cơ bản Google Analytics)
  3. Một vài các tính năng nâng cao của Google Analytics (Phần 3: Google Analytics nâng cao)
1. Tổng quan về Google Analytics
Google Analytisc là một công cụ phân tích các dữ liệu của website miễn phí, được cung cấp bởi Google. Các dữ liệu này sẽ xoay quanh các hành vi của người dùng trước, trong và sau khi truy cập trang.
Từ những số liệu này, Google Analytics sẽ cho bạn một cái nhìn tổng quan về việc:
  • Người dùng truy cập trang bằng cách nào
  • Mức độ và loại tương tác của người dùng trên trang
  • Cảm nhận của họ về nội dung trên trang
  • Tại sao họ lại không thực hiện mua hàng trên trang
  • Đối với các marketer, thì giải đáp được những câu hỏi này tức là bạn đã có thể biết được lý do tại sao tỷ lẹ chuyển đổi (conversion rate) thấp và cách gia tăng tỷ lệ này.
Những số liệu mà Google Analytics thu được bắt nguồn từ mã tracking code được cài đặt trên trang (mã bạn nhận được khi cài đặt Google Analytics).
Như vậy, những trang không có Analytics sẽ không thể gửi dữ liệu về server lưu trữ và xử lý số liệu của Google.

2. Hướng dẫn sử dụng giao diện Google Analytics

Để sử dụng thành thạo công cụ này trên Google Analytics, bạn cần nắm bắt được những thao tác cơ bản sau:
  • thiết lập và tùy chỉnh các mốc thời gian
  • đọc và tùy chỉnh các bảng số liệu của Google Analytics, lọc dữ liệu và thay đổi chế độ hiển thị của báo cáo trong Google Analytics như (bảng biểu, biểu đồ cột, tròn,..)
  • phân tích luồng lưu lượng truy cập đến website

3. Một vài các tính năng nâng cao của Google Analytics

  • Goal – mục tiêu: Công cụ 
    Để hỗ trợ các quản trị viên thiết lập và quản lý số các chuyển đổi trên trang, Google Analytics sử dụng công cụ Goal để tạo lập các mục tiêu trên trang, từ đó giúp các quản trị viên tính toán số chuyển đổi trên trang. (một chuyển đổi sẽ được xác nhận khi có một mục tiêu được hoàn thành bởi người dùng)
  • Kênh:
    Giúp mô phỏng và quản lý chu trình hành vi của người dùng khi truy cập website (kênh mô phỏng phiều Marketing – Marketing Funnel)
    Đây là kênh mô phỏng phiều Marketing (Marketing Funnel). Đồng thời cung cấp cho người dùng các công cụ hỗ trợ để quản lý các hành vi của người dùng trong toàn bộ kênh, ngay từ khi người dùng mới chỉ là khách vào thăm cho đến khi họ trở thành khách hàng của website.

Featured Post

Big Data : Nhu cầu thị trường - Định hướng nghề nghiệp - Tính chất công việc

Tập hợp một số câu hỏi từ bạn trên page  và từ các buổi thuyết trình ở Barcamp 1. Các người làm việc tro...