ai

Def : Artificial intelligence, or AI, is technology that enables computers and machines to simulate human intelligence and problem-solving capabilities.

Machine Learning and Deep Learning disciplines involve the development of AI algorithms, modeled after the decision-making processes of the human brain, that can ‘learn’ from available data and make increasingly more accurate classifications or predictions over time.

Artificial intelligence has gone through many cycles of hype, but even to skeptics, the release of ==ChatGPT== seems to mark a turning point. The last time generative AI loomed large in computer vision, but now the leap forward is in ==Natural Language Processing (NLP)==.

Today, generative AI can learn and synthesize not just human language but other data types including images, video, software code, and even molecular structures.

AI vs ML vs DL vs Neural Network

AI is the overarching system.

Machine learning is a subset of AI.

Deep learning is a subfield of machine learning

==AI== > ==ML== > ==DL==

Both machine learning and deep learning algorithms use neural networks to ‘learn’ from huge amounts of data.

Neural networks make up the backbone of deep learning algorithms.

==Neural Network==

A neural network is a machine learning program, or model, that makes decisions in a manner similar to the human brain, by using processes that mimic the way biological neurons work together to identify phenomena, weigh options and arrive at conclusions.

They consist of layers of interconnected nodes that extract features from the data and make predictions about what the data represents.

Neural networks, or artificial neural networks (ANNs), are comprised of node layers, containing an input layer, one or more hidden layers, and an output layer. Each node, or artificial neuron, connects to another and has an associated weight and threshold. If the output of any individual node is above the specified threshold value, that node is activated, sending data to the next layer of the network. Otherwise, no data is passed along to the next layer of the network by that node.

Neural networks rely on training data to learn and improve their accuracy over time. Once they are fine tuned for accuracy, they are powerful tools in computer science and artificial intelligence, allowing us to classify and cluster data at a high velocity.

Tasks in speech recognition or image recognition can take minutes versus hours when compared to the manual identification by human experts. One of the best-known examples of a neural network is Google’s search algorithm.

[!important]
The perceptron is the oldest neural network, created by Frank Rosenblatt in 1958.

==Neural Network== & ==Deep Learning==

The “deep” in deep learning is just referring to the number of layers in a neural network. A neural network that consists of more than three layers—which would be inclusive of the input and the output—can be considered a deep learning algorithm or a deep neural network.

A neural network that only has three layers is just a basic neural network.

Neural Networks are a ==subset of machine learning==, and at the ==heart of deep learning== models.

![[1000044781.png]]

[[What is ML]]

==Machine Learning== is focuses on the using data and algorithms to enable AI to imitate the way that humans learn, gradually improving its accuracy.

==ML== vs ==DL==

==Machine learning== and ==Deep learning== differ in the types of neural networks they use, and the amount of human intervention involved.

Deep learning eliminates some of data pre-processing that is typically involved with machine learning

These algorithms can ingest and process unstructured data, like text and images, and it automates feature extraction, removing some of the dependency on human experts.

For example, let’s say that we had a set of photos of different pets, and we wanted to categorize by “cat”, “dog”, “hamster”, et cetera. Deep learning algorithms ==can determine== which features (e.g. ears) are most important to distinguish each animal from another. In machine learning, this hierarchy of features is established ==manually by a human expert==.

Algorithms

==Classic machine learning== ==algorithms== use neural networks with an input layer, one or two ‘hidden’ layers, and an output layer. Typically, these algorithms are limited to supervised learning:

==Deep learning algorithms== use deep neural networks—networks composed of an input layer, three or more (but usually hundreds) of hidden layers, and an output layout. These multiple layers enable unsupervised learning:

Method

Supervised machine learning

The data needs to be structured or labeled by human experts to enable the algorithm to extract features from the data.

Supervised learning helps organizations solve a variety of real-world problems at scale, such as classifying spam in a separate folder from your inbox.

Unsupervised machine learning

They automate extraction of features from large, unlabeled and unstructured data sets. Because it doesn’t require human intervention, ==deep learning== essentially enables machine learning at scale.

Reinforcement machine learning

Reinforcement machine learning is a machine learning model that is similar to supervised learning, but the algorithm isn’t trained using sample data.

This model learns as it goes by using trial and error. A sequence of successful outcomes will be reinforced to develop the best ==recommendation== or policy for a given problem.

Pros and cons of ML

Pros :

1. Identifies patterns and trends in large datasets that humans might miss.
2. Requires minimal human intervention for analysis.
3. Continuously improves algorithms with more data input.
4. Enhances personalized customer experiences based on individual interactions.

Cons :

1. Requires large, accurate, and unbiased training datasets.
2. Data collection and processing can be resource-intensive.
3. Prone to errors with small or biased samples.
4. Risk of producing misleading results if the data quality is poor.
5. Organizations should only act on results with high confidence in the data and output.

==Challenges of Machine learning and Deep Learning==

Technological singularity

• Many researchers are not concerned with the idea of AI surpassing human intelligence in the near future.
• Technological singularity is also referred to as strong AI or superintelligence.

Philosopher Nick Bostrum defines superintelligence as

Any intellect that vastly outperforms the best human brains in practically every field, including scientific creativity, general wisdom, and social skills.

AI impact on jobs

• Artificial intelligence will shift the demand for jobs to other areas. There will need to be individuals to help manage AI systems. There will still need to be people to address more complex problems

Privacy

• Privacy is a myth

Bias and discrimination

• Amazon unintentionally discriminated against job candidates by gender for technical roles, and the company ultimately had to scrap the project.

Accountability

Hardware requirements

• Deep learning requires a tremendous amount of computing power. High performance graphical processing units (GPUs) are ideal because they can handle a large volume of calculations in multiple cores with copious memory available.

![[6f944150-5656-4e80-a5a1-db0c0da19d7e-1_all_47109.jpg]]

Types of artificial intelligence: weak AI vs strong AI

• ==Artificial Narrow Intelligence (ANI)==
• ==Artificial General Intelligence (AGI)==
• ==Artificial Super Intelligence (ASI)==

Weak AI—also known as narrow AI or ==Artificial narrow intelligence (ANI)==—is AI trained and focused to perform specific tasks. it enables some very robust applications, such as Apple’s Siri, Amazon’s Alexa, IBM Watsonx™, and self-driving vehicles.

Strong AI is made up of ==Artificial general intelligence (AGI)== & ==Artificial super intelligence (ASI)==

==AGI, or general== ==AI==, is a ==theoretical== form of AI where a machine would have an intelligence equal to humans; it would be self-aware with a consciousness that would have the ability to solve problems, learn, and plan for the future.

==ASI—also known as superintelligence==—would ==surpass== the intelligence and ability of the ==human brain==. While strong AI is still entirely ==theoretical== with no practical examples in use today, that doesn’t mean AI researchers aren’t also exploring its development.

In the meantime, the best examples of ASI might be from science fiction, such as ==HAL== 9000, the superhuman and rogue computer assistant in ==2001: A Space Odyssey====.==

![[1000044780.jpg]]

Natural language processing (NLP)

NLP enables computers and digital devices to recognize, understand and generate text and speech by combining computational linguistics—the rule-based modeling of human language—together with statistical modeling, machine learning (ML) and deep learning.

NLP research has enabled the era of generative AI, from the communication skills of large language models (LLMs) to the ability of image generation models to understand requests.

The rise of generative models

Generative AI refers to deep-learning models that can take raw data—say, all of Wikipedia or the collected works of Rembrandt—and “learn” to generate statistically probable outputs when prompted. At a high level, generative models encode a simplified representation of their training data and draw from it to create a new work that’s similar, but not identical, to the original data.

Generative models have been used for years in statistics to
analyze numerical data. The rise of deep learning, however, made it possible to extend them to images, speech, and other complex data types. Among the first class of AI models to achieve this cross-over feat were variational autoencoders, or VAEs, introduced in 2013. VAEs were the first deep-learning models to be widely used for generating realistic images and speech.

Early examples of models, including GPT-3, BERT, or DALL-E 2, have shown what’s possible. In the future, models will be trained on a broad set of unlabeled data that can be used for different tasks, with minimal fine-tuning.

Artificial intelligence applications

1. Speech recognition

Also known as automatic speech recognition (ASR), computer speech recognition, or speech-to-text, speech recognition uses NLP to process human speech into a written format.(==Pixel Record App==) Many mobile devices incorporate speech recognition into their systems to conduct voice search—Hey Google, Siri, Alexa for example—or provide more accessibility around texting in English or many widely-used languages.

1. Customer service

Online virtual agents and chatbots are replacing human agents along the customer journey. They answer frequently asked questions (FAQ) around topics, like shipping, or provide personalized advice, cross-selling products or suggesting sizes for users, changing the way we think about customer engagement across websites and social media platforms. Examples include messaging bots on e-commerce sites with virtual agents , messaging apps, such as Slack and Facebook Messenger, and tasks usually done by virtual assistants and voice assistants.

1. Computer vision

This AI technology enables computers and systems to derive meaningful information from digital images, videos and other visual inputs, and based on those inputs, it can take action. This ability to provide recommendations distinguishes it from image recognition tasks. Powered by convolutional neural networks, computer vision has applications within photo tagging in social media, radiology imaging in healthcare, and self-driving cars within the automotive industry.

1. Supply chain

Adaptive robotics act on Internet of Things (IoT) device information, and structured and unstructured data to make autonomous decisions. NLP tools can understand human speech and react to what they are being told. Predictive analytics are applied to demand responsiveness, inventory and network optimization, preventative maintenance and digital manufacturing. Search and pattern recognition algorithms—which are no longer just predictive, but hierarchical—analyze real-time data, helping supply chains to react to machine-generated, augmented intelligence, while providing instant visibility and transparency.

1. Weather forecasting

The weather models broadcasters rely on to make accurate forecasts consist of complex algorithms run on supercomputers. Machine-learning techniques enhance these models by making them more applicable and precise.

1. Anomaly detection

AI models can comb through large amounts of data and discover atypical data points within a dataset. These anomalies can raise awareness around faulty equipment, human error, or breaches in security. Fraud detection: Banks and other financial institutions can use machine learning to spot suspicious transactions. Supervised learning can train a model using information about known fraudulent transactions. Anomaly detection can identify transactions that look atypical and deserve further investigation.

1. Recommendation engines:

Using past consumption behavior data, AI algorithms can help to discover data trends that can be used to develop more effective cross-selling strategies. Recommendation engines are used by online retailers to make relevant product recommendations to customers during the checkout process.

1. Automated stock trading:

Designed to optimize stock portfolios, AI-driven high-frequency trading platforms make thousands or even millions of trades per day without human intervention.

1. Robotic process automation (RPA):

Also known as software robotics, RPA uses intelligent automation technologies to perform repetitive manual tasks.

[[History of artificial intelligence-]]

Source : Infosys Spring Board

• In 1959, Arthur Samuel defined machine learning as “A field of study that gives computers the ability to learn without being explicitly programmed”.

• IBM created a program that could play chess, called Deep Blue. The chess grand master, Gary Kasparov competed with Deep Blue in 1996 and 1997 respectively in 2 games of chess. The results of both the games are as shown below.-

![[1000053269.jpg]]

Supervised machine learning model:

1. In the first step, we train the machine with known data so that it learns something from it.In the
2. second step we expect the machine to utilize the knowledge it gained in the previous step and classify a new unknown data point.
3. In the third step, the model is evaluated on the basis of how accurately it has classified the unknown data.

Supervised machine learning model: Types

1. Classification: Used to predict discrete results.
2. Regression: Used to predict continuous numeric results

Unsupervised machine learning model – Clustering

Semi-supervised machine learning

• Step 1: Train the model with labeled data points only.
• Step 2: Use the above model to predict the labels of the unlabeled data points
• Step 3: Combine the existing labeled data points with the newly labeled data points and use it to retrain the model
• Step 4: Repeat the 2nd and 3rd steps until it converges

Applications of semi-supervised learning are text processing, video-indexing, bioinformatics, web page classification and news classification among others

Reinforcement learning

Reinforcement machine learning algorithm is a reward based and immediate feedback technique.

![[/1000053270 2.png|1000053270 2.png]]

Tools and packages for Data Science

![[/1000053271 2.png|1000053271 2.png]]

![[/1000053272 2.png|1000053272 2.png]]

![[/1000053273 2.png|1000053273 2.png]]

![[/1000053274 2.png|1000053274 2.png]]

Data Science implementation - Business use case

Problem

Country Bank of India wants to cut down on their losses due to bad loans. It approaches a data analytics firm to help them reduce these losses by X%.

Solution

Step 1: Define the goal

• Why is the project being started?
• What is missing currently and what exactly is required?
• What are they currently doing to fix the problem, and why isn’t it working?
• What all resources are needed?
• What kind of data is available? Is domain expertise available within the team?
• What are the computational resources available/required?
• How does the business organization plan to deploy the derived results?
• What kind of problems need to be addressed for successful deployment?

Step 2: Collect and manage data

• What all data is available?
• Will it help in solving the problem?
• Is the data enough to carry out analysis?
• Is the quality of data up to the mark?

Step 3: Build a model

Classification: Determining which among the given categories a data point falls under

Scoring: Predicting or estimating a quantifiable value

Ranking: Ordering the data points depending on the priorities involved

Clustering: Grouping similar items based on certain parameters

Finding relations: Finding associations between various features of the data

Characterization: Creating plots, graphs and various reports for understanding the data better

Step 4: Model evaluation

• Is the model accurate enough for our needs?
• Does the model meet the expectations?
• Is it better than the methodology being currently used?

Step 5: Present results and document

At this stage we have achieved a desirable model. A model that meets all the requirements and goals we set for ourselves at the beginning of the project. The next step is to showcase the project to various audience as follows:

• Present the details of the model to all the collaborators, clients and sponsors.
• Provide everyone in charge of usage and maintenance of the model, once deployed, with documentation that covers all aspects of the working of the model.

Step 6: Deploy model

The last and final step is to deploy the model. Usually from this point ahead the data scientist is no longer associated with the operations of the model. But before they are off the job they must make sure that the following are in place:

• The model has been tested thoroughly and generalizes well.
• The model should be able to adjust well to unforeseen environmental changes.
• The model has been deployed in a pilot program and any problems that cropped up in the last moment were taken care of by updating the model accordingly.

Characteristics of a successful Data Science project

![[/1000053280 2.png|1000053280 2.png]]

![[/1000053281 2.png|1000053281 2.png]]

Churn Prediction

Churn implies loss of customers to competition. For any company, it costs more to acquire new customers than to retain the old ones. As churn prediction aids in customer retention, it is extremely important especially for businesses with a repeat customer base. The application of this model cuts across domains such as Banking, E-Retail, Telecom, Energy and Utilities.

Sentiment Analysis

Also referred to as opinion mining, it is the process of computationally identifying what customers like and dislike about a product or a brand. A domain which relentlessly makes use of sentiment analysis is the Retail industry. Companies like Amazon, Flipkart, Reliance, Paytm use customer feedback from social networking sites like Facebook, Twitter, etc. or their own company websites to find out what their customers are talking about and how they feel i.e. positive, negative, or neutral. They leverage this information to reposition their products and provide better/new services.

Online Advertisement

The incremental growth in the complexity of ads industry is due to the ease of access to the internet via a wide variety of devices around the world. This gives the advertisers an opportunity to study user preferences and online trends. The insights offered to them through these analysis, translates to actionable items on issues and opportunities such as reducing ad-blindness or optimizing cost-per-action (CPA) and click-through-rates (CTR).

Recommendations

Many e-retail companies like Amazon, Netflix, Spotify, Best Buy, You Tube among many others use recommender systems to improve a customer’s shopping experience. This offers the companies a chance to gather information on customer’s preferences, purchases and other browsing patterns which lend insights that can amplify their return on investment.

Truth and Veracity

In today’s digital world, the quantity of fake news is on the rise. Not only does vast majorities of population fall prey to misinformation but it effects businesses negatively. Data Science is being used to ensure data veracity or in other words, verify the truthfulness of data based on both accuracy and context. Companies such Facebook, Twitter, Starbucks, Costco and many others are combating fake news currently with the help of various data science techniques.

News Aggregation

A news aggregator gathers and clusters stories of the same topic from several leading news websites and also traces the genuine source of a news item and what is the course of the story. It has a special interactive timeline that allows the reader to flip swiftly between headlines, refining their search by country or by specific news sites. Notable examples include Google News, Reddit, Flipboard, Pulse etc.

Scalability

Scalability refers to an enterprise’s ability to handle increased demands. In the corporate environment, a scalable company is one that can maintain or improve profit margins while sales volume increases. Many a times the process is slowed down by human intervention for decisions. For example, the credit operations in banks invest substantial time in assessing the credit worthiness of a client. Client management teams take long time to suggest the right product to the customer/suggest alternatives. The client help desk takes long time to provide the desired info to the client. If these processes can be automated, the business can scale up. Data science helps build systems like recommender systems, Chabot’s etc. to achieve scalability.

Content Discovery/ Search

Content discovery involves using predictive algorithms to help make content recommendations to users based on how they search. Search engines such as Google, Bing and Yahoo and various other platforms are now using intelligent learning mechanisms to understand user preferences to be able to suggest content that’s most suitable for them.
Few more platforms that use content discovery algorithms are Facebook and YouTube. The content that appears in an individual’s Facebook news feed and the videos that appear in the “Recommended for You” section of YouTube user’s account, are both altered according to each user’s past behavior and personal preferences.

Intelligent Learning

Intelligent learning has become a part of our day to day lives in various forms. For example, Google Maps uses undesignated location data from various smart devices to predict the flow of traffic in real time. It also utilizes user based reports on incidents that might affect the traffic, like road construction and accidents, to help suggest fastest routes for travel, to users.
Another example would be ride sharing apps like Uber and Ola. They optimize the ride experience by not only minimizing the ride time but also by matching users with other passengers for least amount of detours in shared rides
Other examples of intelligent learning include self-driving cars, smart-email categorization, credit-card fraud detection, etc.

Personalized Medicine

In many cases, the success of a particular treatment for a patients’ condition cannot be predicted beforehand. Thus, many medical practitioners follow a non-optimal trial-and-error approach.
In personalized medicine, a doctor needs to study an individual’s genes, environment and lifestyle. This would help tailor treatments for specific medical conditions as opposed to a trial and error approach. This would also enable pharmaceutical researchers to create combination drugs targeting a specific genomic profile which in turn increases safety and efficiency.
Companies that are active in the field of personalized medicine are
Roche, Novartis, Johnson & Johnson among others.

![[/1000053282 2.png|1000053282 2.png]]

![[/1000053283 2.png|1000053283 2.png]]

BooksPractical Data Science with R by Nina Zumel, John Mount, Jim PorzakThink Stats - Probability and Statistics for Programmers by Allen B. Downey, Green Tea Press, Needham, Massachusetts