Module-1.1.pdf of aiml engineering mod 1

FPPT.com
Heena-Kouser-5th-SEM-AI&ML-17CS81 1
Module 1
Introduction

• Module-1 Introduction: What is AI? Foundations and History of AI
Problem‐solving: Problem‐solving agents, Example problems,
Searching for Solutions, Uninformed Search Strategies: Breadth First
search, Depth First Search.
• Module-2 Informed Search Strategies: Greedy best-first search,
A*search, Heuristic functions. Introduction to Machine Learning ,
Understanding Data
• Module-3 Basics of Learning theory Similarity Based Learning
Regression Analysis
• Module-4 Decision Tree learning Bayesian Learning
• Module-5 Artificial neural Network Clustering Algorithms
2

Artificial Intelligence and Machine Learning
• Course Learning Objectives
• CLO 1. Gain a historical perspective of AI and its foundations
• CLO 2. Become familiar with basic principles of AI toward
problem solving
• CLO 3. Familiarize with the basics of Machine Learning &
Machine Learning process, basics of Decision Tree, and
probability learning
• CLO 4. Understand the working of Artificial Neural Networks
and basic concepts of clustering algorithms
3

• At the end of the course the student will be able to:
• CO 1. Apply the knowledge of searching and reasoning
techniques for different applications.
• CO 2. Have a good understanding of machine leaning in
relation to other fields and fundamental issues and
challenges of machine learning.
• CO 3. Apply the knowledge of classification algorithms
on various dataset and compare results
• CO 4. Model the neuron and Neural Network, and to
analyze ANN learning and its applications.
• CO 5. Identifying the suitable clustering algorithm for
different pattern
4

• Textbooks 1. Stuart J. Russell and Peter
Norvig, Artificial Intelligence, 3rd
Edition, Pearson,2015
• 2. S. Sridhar, M Vijayalakshmi “Machine
Learning”. Oxford ,2021
5

Module-1 Introduction: What is AI?
• AI is exciting.
• but we have not said what it is.
• In Figure 1.1 we see eight definitions of
AI, laid out along two dimensions.
• The definitions on top are concerned
with thought processes and reasoning,
whereas the ones on the bottom
address behavior.
6

• The definitions on the left measure
success in terms of fidelity to human
performance, whereas RATIONALITY
the ones on the right measure against
an ideal performance measure, called
rationality.
• Asystem is rational if it does the “right
thing,” given what it knows.
7

• The Turing Test, proposed by Alan Turing TURING TEST
(1950), was designed to provide a satisfactory
operational definition of intelligence.
• A computer passes the test if a human interrogator,
after posing some written questions, cannot tell
whether the written responses come from a person or
from a computer.
• The test involves a human judge engaging in natural
language conversations with a machine and a human,
without knowing which is which. If the judge cannot
reliably distinguish between the machine and the
human based on their responses, then the machine is
said to have passed the Turing Test.
9
Acting humanly: The Turing Test approach

• Here's a simplified example:
• Imagine a scenario where a human judge communicates with two entities, one human (H) and
one machine (M), through a text-based interface without knowing which is which. The
conversation might look like this:
• Judge: Hello! How are you today?
• Entity 1: I'm doing well, thank you. How about you?
• Entity 2: I'm functioning within normal parameters. How about yourself?
• Judge: Interesting responses. What do you enjoy doing in your free time?
• Entity 1: I love reading books and going for hikes.
• Entity 2: I don't have free time as I'm always ready to assist with tasks.
• Judge: Hmm, both seem plausible. Can you tell me a joke?
• Entity 1: Sure, here's one: Why did the scarecrow win an award? Because he was outstanding in
his field!
• Entity 2: I'm not programmed for humor, but I can provide information on a wide range of topics.
• Judge: This is challenging; both entities are giving reasonable answers. Which one is the machine?
• If the judge cannot consistently identify the machine, then the machine is considered to have
passed the Turing Test in this context. It's important to note that passing the Turing Test doesn't
necessarily mean that a machine possesses true understanding or consciousness; it only means
that it can simulate human-like responses well enough to deceive a human judge in a
conversation.
10

• The computer would need to possess the following
capabilities.
• natural language processing to enable it to
communicate successfully in English;
• knowledge representation to store what it knows
or hears;
• automated reasoning to use the stored
information to answer questions and to draw new
conclusions;
• machine learning to adapt to new circumstances
and to detect and extrapolate patterns.
11

• To pass the total Turing Test, the computer will need
• COMPUTER VISION to perceive objects, and
• ROBOTICS to manipulate objects and move about.
• These six disciplines compose most of AI, and Turing deserves
credit for designing a test that remains relevant 60 years later.
• Yet AI researchers have devoted little effort to passing the
Turing Test, believing that it is more important to study the
underlying principles of intelligence than to duplicate an
exemplar.
• The quest for “artificial flight” succeeded when the Wright
brothers and others stopped imitating birds and started using
wind tunnels and learning about aerodynamics.
12

Thinking humanly: The cognitive modeling approach
• A cognitive model is a representation or
framework that describes the mental processes
and structures involved in human cognition,
including perception, memory, learning,
problem-solving, and decision-making.
• There are three ways to do this:
• Through introspection—trying to catch our own
thoughts as they go by;
• Through psychological experiments—observing a
person in action;
• Through brain imaging —observing the brain in
action.
13

14
Thinking rationally: The “laws of
thought” approach

History of AI: The gestation of
artificial intelligence (1943–1955)
• Artificial Intelligence (AI) is not the invention of a single person; rather, it
has evolved over time through the contributions of numerous researchers,
scientists, and engineers.
• Warren McCulloch and Walter Pitts, in 1943, are credited with the first
work recognized as Artificial Intelligence (AI). Their model of artificial
neurons drew on three key sources:
• Physiology of Neurons:
– They based their work on knowledge of the basic physiology and
function of neurons in the brain.
• Formal Logic:
– They incorporated a formal analysis of propositional logic, influenced
by the work of Russell and Whitehead.
• Turing's Theory of Computation:
– They also integrated Alan Turing's theory of computation into their
model.
15

• Key Features of McCulloch and Pitts' Model:
• Artificial Neurons:
– They proposed a model of artificial neurons where each neuron could be either "on"
or "off."
• Neuron Activation:
– Neurons would switch to an "on" state in response to stimulation by a sufficient
number of neighboring neurons.
• Representation of Neuron State:
– The state of a neuron was considered "factually equivalent to a proposition proposing
its adequate stimulus."
• Computational Universality:
– They demonstrated that any computable function could be computed by some
network of connected neurons.
• Logical Connectives:
– McCulloch and Pitts showed that logical connectives (and, or, not, etc.) could be
implemented by simple network structures.
• Learning Capability:
– They suggested that suitably defined networks could learn.
• Donald Hebb's Contribution:
• Hebbian Learning:
– Donald Hebb, in 1949, introduced a simple updating rule for modifying the connection
strengths between neurons, known as Hebbian learning.
16

• Building the First Neural Network Computer
(1950):
• Marvin Minsky and Dean Edmonds,
undergraduate students at Harvard, built the
first neural network computer called SNARC
in 1950.
• SNARC used 3000 vacuum tubes and a surplus
automatic pilot mechanism from a B-24
bomber to simulate a network of 40 neurons.
17

1.3.2 The birth of artificial
intelligence (1956)
• 1. John McCarthy's Move to Stanford and Dartmouth:
• After receiving his Ph.D. from Princeton in 1951, John McCarthy moved to Stanford
and later to Dartmouth College.
• Dartmouth College became the official birthplace of the field of Artificial Intelligence
(AI).
• 2. Dartmouth Workshop (1956):
• McCarthy, along with Marvin Minsky, Claude Shannon, and Nathaniel Rochester,
organized a two-month workshop at Dartmouth in the summer of 1956.
• The workshop aimed to bring together U.S. researchers interested in automata
theory, neural nets, and the study of intelligence.
• 3. Proposal for the Workshop:
• The proposal suggested a 2-month, 10-man study of artificial intelligence at
Dartmouth in 1956.
• The underlying conjecture was that every aspect of learning or any other feature of
intelligence could be precisely described, allowing a machine to simulate it.
• Goals included making machines use language, form abstractions and concepts, solve
human-reserved problems, and improve themselves.
18

• 4. Attendees at the Workshop:
• There were 10 attendees, including Trenchard More from Princeton,
Arthur Samuel from IBM, and Ray Solomonoff and Oliver Selfridge from
MIT.
• 5. Carnegie Tech Researchers:
• Allen Newell and Herbert Simon from Carnegie Tech (now Carnegie Mellon
University) were notable attendees.
• Newell and Simon presented a reasoning program called the Logic Theorist
(LT) at the workshop.
• 6. Newell and Simon's Contribution:
• Newell and Simon's Logic Theorist was a reasoning program capable of
thinking non-numerically.
• Simon claimed that they had solved the "venerable mind–body problem"
by inventing a computer program with this capability.
• In summary, the key points include John McCarthy's role in establishing
Dartmouth College as the birthplace of AI, the organization of the
influential Dartmouth workshop in 1956, and the notable contributions of
attendees like Allen Newell and Herbert Simon, who presented the Logic
Theorist.
19

– In the early years of AI, there were notable
successes despite the limitations of primitive
computers and programming tools.
– Given the perception that computers were initially
seen as capable only of arithmetic, any
demonstration of cleverness was considered
remarkable.
• 2. "Look, Ma, No Hands!" Era:
– John McCarthy referred to this period as the "Look,
Ma, no hands!" era, highlighting the astonishment at
what computers could achieve despite minimal
expectations.
20
Early enthusiasm, great
expectations (1952–1969)
• 1. Early Successes in AI:

• 3. Newell and Simon's Contributions:
– Newell and Simon achieved early success with the Logic
Theorist, a reasoning program.
– They followed up with the General Problem Solver (GPS),
designed to imitate human problem-solving protocols.
– GPS considered subgoals and possible actions in a way similar
to how humans approached problems.
• 4. Thinking Humanly Approach:
– GPS was one of the first programs to embody the "thinking
humanly" approach, designed to simulate human problem-
solving strategies.
• 5. Physical Symbol System Hypothesis:
– Newell and Simon formulated the Physical Symbol System
Hypothesis, stating that "a physical symbol system has the
necessary and sufficient means for general intelligent action."
– This hypothesis asserts that any system, whether human or
machine, exhibiting intelligence must operate by manipulating
data structures composed of symbols.
21

• 6 AI Programs at IBM:
– Nathaniel Rochester and his colleagues at IBM were among the early
contributors to AI programs.
– Herbert Gelernter, in 1959, constructed the Geometry Theorem
Prover, a program capable of proving challenging theorems in
geometry.
• 7 Arthur Samuel's Checkers Program:
– Starting in 1952, Arthur Samuel wrote a series of programs for
checkers (draughts).
– Samuel's checkers program eventually learned to play at a strong
amateur level through a learning process.
– The program disproved the idea that computers can only do what
they are explicitly told to do.
– Samuel's program surpassed its creator's playing ability,
demonstrating the capacity for machines to learn and improve
independently.
– The program was showcased on television in February 1956, leaving a
strong impression.
22

• 8 Computer Time Challenges:
– Similar to Turing, Arthur Samuel faced challenges in
finding computer time.
– Samuel worked at night, utilizing machines that were still
on the testing floor at IBM's manufacturing plant to
develop and test his programs.
• In summary, the key point is the early development of AI
programs at IBM, including Herbert Gelernter's Geometry
Theorem Prover and Arthur Samuel's groundbreaking work on
a checkers program that demonstrated machine learning and
surpassed human-created strategies. The challenges of
accessing computer time during this period are also
highlighted.
23

• 9. Marvin Minsky's Move to MIT (1958):
– In 1958, Marvin Minsky moved to MIT
(Massachusetts Institute of Technology).
– His initial collaboration with John McCarthy did not
last due to differing perspectives on AI.
• 10. McCarthy and Minsky's Different Emphases:
– McCarthy emphasized representation and
reasoning in formal logic.
– Minsky, on the other hand, was more interested in
practical implementation and getting programs to
work.
– Minsky eventually developed an anti-logic outlook,
diverging from McCarthy's emphasis on formal
logic.
24

• 11 . McCarthy's AI Lab at Stanford (1963):
– In 1963, John McCarthy started the AI lab at Stanford University.
– McCarthy's plan was to use logic to build the ultimate Advice Taker.
• 12 . J. A. Robinson's Contribution (1965):
– In 1965, J. A. Robinson made a significant contribution with the
discovery of the resolution method.
– The resolution method is a complete theorem-proving algorithm for
first-order logic (referenced in Chapter 9).
• 13 . Work at Stanford AI Lab:
– The work at Stanford's AI lab emphasized general-purpose methods for
logical reasoning.
– Applications of logic included Cordell Green's question-answering and
planning systems (1969b).
– The Shakey robotics project at the Stanford Research Institute (SRI) was
also part of this initiative.
• In summary, key points include Marvin Minsky's move to MIT, the differing
emphases of McCarthy and Minsky in the field of AI, McCarthy's
establishment of the AI lab at Stanford, J. A. Robinson's contribution to
theorem proving, and the application of logic in projects such as question-
answering systems and the Shakey robotics project at SRI.
25

1.3.4 A dose of reality (1966–1973)
• The statement was made by Herbert Simon in 1957.
1. Timeframe: The statement was made by Herbert Simon in 1957.
2.Claimed Abilities of Machines:
– Machines can think.
– Machines can learn.
– Machines can create.
3. Prediction about Progress:
– The abilities of machines to think, learn, and create are expected to increase
rapidly.
4. Future Vision:
– In a visible future, the range of problems that machines can handle will be as
extensive as the range to which the human mind has been applied.
5. Overall Tone:
– The tone of the statement is confident and forward-looking, expressing the
belief in the rapid advancement of artificial intelligence to match and even
exceed certain cognitive capabilities of the human mind.
Heena-Kouser-5th-SEM-AI&ML-17CS81 26

• In almost all cases, however, these early systems
turned out to fail miserably when tried out on
wider selections of problems and on more difficult
problems.
1. Lack of Background Knowledge in Early AI
Systems:
• Early AI programs often lacked the necessary
background knowledge about the subject matter.
• They succeeded through simple syntactic
manipulations but struggled when faced with a
wider selection of problems.
• The example of machine translation illustrates the
challenges, where accurate translation requires
context and knowledge to resolve ambiguity.
27

2. Intractability of Problems:
• Many early AI programs solved problems by trying
out different combinations of steps until a solution
was found.
• This strategy worked for microworlds with few
objects and short solution sequences.
• However, as problems scaled up, the intractability
became apparent, and the optimistic belief that
scaling up was just a matter of faster hardware and
larger memories was challenged.
28

3. Fundamental Limitations in Basic Structures for
Intelligent Behavior:
• Some fundamental limitations existed in the
basic structures used to generate intelligent
behavior.
• The example of perceptrons (a simple form of
neural network) highlighted their limitations in
representation and learning capacity.
• Minsky and Papert's book "Perceptrons" (1969)
demonstrated that certain simple neural
networks had significant limitations in what they
could represent and learn.
29

1 Weak Methods in Early AI Research:
– Early AI research primarily involved
weak methods, which were general-
purpose search mechanisms attempting
to combine elementary reasoning steps
to find complete solutions.
– Weak methods were called so because,
while general, they struggled to scale
up to large or difficult problem
instances.
30
1.3.5 Knowledge-based systems:
The key to power? (1969–1979)

2. Alternative Approach: Domain-Specific
Knowledge:
– An alternative to weak methods is the
use of more powerful, domain-specific
knowledge.
– Domain-specific knowledge allows
larger reasoning steps and is more
effective in handling typical cases
within narrow areas of expertise.
31

3. DENDRAL Program:
– The DENDRAL program, developed at
Stanford, exemplified the use of
domain-specific knowledge in problem-
solving.
– The goal of DENDRAL was to infer
molecular structure from mass
spectrometer data.
32

4. Problem and Solution in DENDRAL:
– The input to DENDRAL included the
elementary formula of the molecule and
the mass spectrum generated by
bombarding it with an electron beam.
– The naive version of the program
generated all possible structures consistent
with the formula, which was intractable for
moderate-sized molecules.
– The DENDRAL researchers consulted
analytical chemists and found that they
worked by identifying well-known patterns
of peaks in the spectrum.
33

5. Analytical Approach in DENDRAL:
– Analytical chemists used known patterns
of peaks to suggest common
substructures in the molecule.
– Specific rules, like the one for
recognizing a ketone subgroup, were
developed based on the analysis of peak
patterns.
34

6. Importance of Prior Knowledge:
– To solve a hard problems, it is
beneficial to have almost prior
knowledge of the answer.
– The DENDRAL program's success was
attributed to incorporating domain-
specific knowledge from analytical
chemists.
35

• The first successful commercial expert
system, R1, started operating at Digital
Equipment Corporation (DEC) in 1982.
• R1 was designed to assist in configuring
orders for new computer systems.
36
1.3.6 AI becomes an industry (1980–
present)
1. Introduction of R1 Expert System:

2. Success and Proliferation of Expert Systems:
– By 1986, R1 was estimated to save Digital
Equipment Corporation (DEC) approximately
$40 million per year.
– DEC's AI group had 40 expert systems
deployed by 1988, and DuPont had 100 in
use and 500 in development, saving an
estimated $10 million a year.
– Many major U.S. corporations had their AI
groups, and the use or investigation of
expert systems was widespread.
37

3. International AI Developments:
– In 1981, Japan announced the "Fifth
Generation" project, a 10-year plan to build
intelligent computers running Prolog.
– In response, the United States established
the Microelectronics and Computer
Technology Corporation (MCC) to ensure
national competitiveness, with a focus on
AI, chip design, and human-interface
research.
38

4. Ambitious Projects and Industry Growth:
– The AI industry experienced significant
growth, with the overall industry value
increasing from a few million dollars in
1980 to billions of dollars in 1988.
– The industry included hundreds of
companies working on expert systems,
vision systems, robots, and specialized
software and hardware.
39

5 AI Winter:
– Following the period of rapid growth,
there was a subsequent period referred to
as the "AI Winter."
– During this time, many companies faced
challenges and were unable to deliver on
extravagant promises, leading to a
downturn in the AI industry.
6. Global Impact:
– While the U.S., Japan, and Britain all
invested in AI projects, these initiatives did
not meet their ambitious goals.
40

1.3.7.The return of neural networks
(1986–present)
1 Reinvention of Back-Propagation
Algorithm in the Mid-1980s:
• In the mid-1980s, at least four different
groups independently rediscovered the
back-propagation learning algorithm,
which was originally discovered in 1969 by
Bryson and Ho.
• This algorithm was applied to various
learning problems in computer science and
psychology.
41

2. Connectionist Models and the Parallel
Distributed Processing Collection (1986):
• Connectionist models of intelligent
systems emerged in the mid-1980s, using
the back-propagation algorithm.
• The results were widely disseminated in
the collection "Parallel Distributed
Processing" (Rumelhart and McClelland,
1986), generating excitement in the field.
42

3. Competition Between Connectionist and
Symbolic Models:
• Connectionist models were seen by some
as direct competitors to symbolic models
promoted by Newell and Simon and the
logicist approach of McCarthy and others.
• There was a debate about the role of
symbol manipulation in detailed models of
cognition, with some connectionists
questioning its explanatory power.
43

4. Complementary Nature of Connectionist and
Symbolic Approaches:
• Despite initial competition, the current view is
that connectionist and symbolic approaches
are complementary rather than mutually
exclusive.
• Modern neural network research has
bifurcated into two fields: one focused on
creating effective network architectures and
algorithms with an emphasis on mathematical
properties, and the other focused on detailed
modeling of the empirical properties of actual
neurons and neuron ensembles.
44

1.3.8 AI adopts the scientific
method (1987–present)
• Hidden Markov Models (HMMs):
• Probabilistic Reasoning and Decision
Theory:
• Data Mining Techniques:
• Neural Networks:
45

– Researchers, encouraged by progress in
solving subproblems, have revisited the
"whole agent" problem.
– The work on SOAR by Allen Newell, John
Laird, and Paul Rosenbloom is highlighted
as a notable example of a complete agent
architecture.
46
1.3.9 The emergence of intelligent
agents (1995–present)
1. Complete Agent Architecture and SOAR:

2. AI in Web-based Applications:
– AI systems, especially intelligent agents,
have become common in web-based
applications, and the use of "bots" in
everyday language reflects this trend.
– AI technologies underlie various
internet tools such as search engines,
recommender systems, and website
aggregators.
47

• Reorganization of AI Subfields:
– Building complete agents has led to the
realization that previously isolated
subfields of AI may need reorganization
when integrating their results.
– The perspective of building complete
agents has brought AI closer to other
fields like control theory and
economics, which also deal with
agents.
48

• Debate on the Progress of AI:
– Despite successes, some influential
founders of AI, including John McCarthy,
Marvin Minsky, Nils Nilsson, and Patrick
Winston, express discontent with the
progress of AI.
– They advocate for a return to the original
goals of AI, focusing on creating machines
that can think, learn, and create, often
referred to as Human-Level AI (HLAI) or
Artificial General Intelligence (AGI).
49

1.3.9 The emergence of intelligent
agents (1995–present)
• AI systems have become so common in Web-based applications that
the “-bot” suffix has entered everyday language.
• Moreover, AI technologies underlie many Internet tools, such as
search engines, recommender systems, and Web site aggregators.
• They think that AI should put less emphasis on creating ever-
improved versions of applications that are good at a specific task,
such as driving a car, playing chess, or recognizing speech.
• Instead, they believe AI should return to its roots of striving for, in
Simon’s words, “machines that think, that learn and that create.”
They call the effort human-level HUMAN-LEVEL AI or HLAI; their
first symposium was in 2004 (Minsky et al., 2004).
• The effort will require very large knowledge bases; Hendler et al.
(1995) discuss where these knowledge bases might come from
50

1.3.10 The availability of very large
data sets (2001–present)
• It makes more sense to worry about the data
and be less picky about what algorithm to
apply.
• This is true because of the increasing
availability of very large data sources: for
example,
• trillions of words of English and billions of
images from the Web (Kilgarriff and
Grefenstette, 2006); or billions of base pairs of
genomic sequences (Collins et al., 2003).
51

• Instead, given a very large corpus of
unannotated text and just the dictionary
definitions of the two senses—“works,
industrial plant” and “flora, plant life”—one
can label examples in the corpus, and from
there bootstrap to learn new patterns that
help label new examples.
52

Module-1.1.pdf of aiml engineering mod 1

More Related Content

Similar to Module-1.1.pdf of aiml engineering mod 1

Recently uploaded

Module-1.1.pdf of aiml engineering mod 1