ARTIFICIAL INTELLIGENCE - A.Y. 2020/2021

(Click here to jump directly to the program of the course)


Disclaimer for ERASMUS students

Erasmus students are welcome. Nonetheless, there are a few, major things that the perspective Erasmus student has to bear in mind in case she/he decides to attend this course:
1) There are NO DISCOUNTS for Erasmus students. All students are required to come through the exam FULLY positively in order to get their credits.
2) The exam is Italian style, meaning that several students usually fail their first attempts (because the exam is difficult) but they are allowed to come back for whole new examinations in the next MONTHS or even years. Erasmus students may have difficulties, thus, since they usually return to their home countries at the end of the semester (which prevents them from re-trying the exam). This implies that they need to be real sure they are willing to take the exam in the first place, and be real sure they are ready to go at 1st attempt.
3) Also, according to point (2), there will be NO EXTRA EXAMS scheduled in the days immediately next to the official exam date(s) just in order to allow Erasmus students to try again: preparing this exam takes time, no recovery from a complete failure can be seriously expected of any student in less than a few (> 2) weeks time.
4) FYI (it is better to know before you choose): historically, statistical outcomes from previous years show that MORE THAN 95% of the Erasmus students enrolled in this subject COULD NOT COME THROUGH the examination.


Exams

The exam is in two mandatory parts (written, oral) plus a non-mandatory part ("project").

Due to the COVID-19 outbreak, until further notice the traditional rules for the AI exam have been modified as follows.

PREAMBLE: organizing the activities online is hard. I kindly request your help me to this end: in case you enlist in an exam session and then, at any later time and for whatever reason, you decide not to take the exam anymore, please *** do email me ASAP *** to let me know you are not going to show (such that I can prepare the tests according to the actual number of examinees).

Both written and oral parts are online via Google Meet.

The written part takes place on the date and time of the official exam session ("appello"). Candidates to take the exam who have regularly enrolled in the exam will receive an email from me at their official email address @studenti.unisi.it a few minutes before the beginning of the test. The email will contain the link to a Google Meet virtual classroom ready to host the exam. ***Do make sure you join the meeting before the official schedule of the exam*** (late-comers may not be allowed to take the exam). I will identify the candidates via ESSE3 (segreteriaonline.unisi). You will be required to follow the general rules fixed by UniSI for written exams (make sure you double check these rules beforehand). In particular, a visual inspection of your room will be carried out before starting the exam, and the door to your room shall remain closed throughout the whole written test. You will be required to keep your webcam tuned on during the exam, and your microphone turned off.

Once ready, I will email to all candidates the link to their written test, that is a Google Form to be filled in. Different tests will possibly be delivered to different candidates. You will have 16 minutes to complete the test from the moment you receive the link. The test will present a first textual field that you are required to fill in with your name and uni-id ("matricola"), followed by 7 quiz-like questions. Each question will have three alternative answers (only one will be correct). Candidates shall select the proper answer. In case you do not know the answer, you are allowed to let the question unanswered. Make sure you submit your form once ready (or, upon my closing call) by clicking on the send/submit button at the bottom of the form. Correct answers score 3 points, wrong answers score -1 point, unanswered answers score 0 points. Admission to the oral is granted with an overall score of 13 points or higher. The score will also contribute forming the final grade of the whole exam, jointly with the outcome of the oral part. In case you fail the written part you need to re-take it on the next available session.

The oral part will take place in the same virtual classroom (or, in another classroom whose link will be emailed to you in case the oral is not scheduled on the same day of the written test). The session is public, anyone interested in attending passively may require me the link and access the classroom. The oral is reserved to those who positively came through the written part. Depending on organizational issues and on the number of candidates, the oral part may take place on the same date of the written part (right after the Instructor has completed checking the outcomes of the written tests) or on another, forthcoming day that will be communicated promptly (orally on Google Meet on the day of the written part, and/or via email). As a strict rule, the oral will be scheduled within a fortnight (two weeks) from the written test. The oral is a conversation (in English) aimed at assessing the degree of understanding of the subject. The candidate may be asked to write (e.g. calculations or algorithms) on paper, to be shown to the examiner digitally (e.g. online via webcam). The oral may revolve around any possible topics covered by the course, to any degree of detail. The slides of the course are self-contained and allow for a complete preparation of the exam, still they require an in-depth and in-full coverage in order to come through the oral part with good marks. The final grade is decided by the Instructor on the basis of the performance offered by the examinee on both the written and oral tests. Once completed positively, the oral exam gives the candidate the right to be set on the record ("verbalizzato") officially right away. Only in case you get the maximum score (30/30) you will be asked if you are willing to take also the third part of the exam (the project) aiming at "30 cum laude". If one fails the oral, or if one does not show at the oral after having been admitted, she/he is required to re-take the whole exam (including the written part) on one of the next available official exam dates.

As for the project, as usual it is reserved (on a free-will basis) to those candidates who scored 30/30 after the oral and whom the Instructor proposes a laude (and, they would like to go for it). Therefore, never under any circumstances ask the Instructor to assign a project in advance. The project is a reduced-scale research activity where, under supervision from the Instructor, the candidate is required to combine some of the techniques covered by the program of the course, carrying out some experiments (including comparisons with established techniques, and some statistical analysis of the results), that will be summarized eventually in the form of a written technical report. The research activity is considered to be completed upon approval from the Instructor. The outcome of the exam is set on the record ("verbalizzato") on the first available official date scheduled to occur after completion and approval of the technical report.

Note 1: make sure you enroll to the official exam session ("appello") you want to take via the Esse3-segreteriaonline online system.
Note 2: the Instructor cannot enroll/set on the record officially any students who do not match the requirements for taking the exam.
Note 3: no extra exam sessions will be created "on demand" in addition to those officially offered by the regular schedule already (they are more than enough, make sure you make/become a good planner of your own time and resources).


Handbooks and bibliography

1. Russell & Norvig, "Artificial Intelligence: A Modern Approach". Prentice Hall, 2010.

2. Duda & Hart, "Pattern Classification and Scene Analysis". J. Wiley, 1973 (equivalent: Duda, Hart & Stork, "Pattern Classification - Second Edition". J. Wiley, 2001).

3. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition". Proc. of IEEE, vol. 77, no. 2, pp. 257--286, February 1989.

4. E. Trentin & M. Gori, "A survey of hybrid ANN/HMM models for automatic speech recognition". Neurocomputing, 37(1/4); 91-126, March 2001.

5. E. Trentin & M. Gori, "Robust Combination of Neural Networks and Hidden Markov Models for Speech Recognition". IEEE Transactions on Neural Networks, vol. 14, n. 6; 1519-1531, Nov. 2003.

6. C. Bishop, "Neural Networks for Pattern Recognition". Oxford University Press, 1995.

7. C. Bishop, "Pattern Recognition and Machine Learning". Springer, 2006.

8. Ian Goodfellow, Yoshua Bengio and Aaron Courville , "Deep Learning". MIT Press, 2016.

9. A. Freno and E. Trentin, "Hybrid Random Fields - A Scalable Approach to Structure and Parameter Learning in Probabilistic Graphical Models". Springer (ISBN 978-3-642-20307-7), 2011.


Office Hours

Under the COVID-19 emergency, office hours can be scheduled online via Google Meet upon appointment. Please email the Instructor for any needs or request of information.


Lecture Hours

Tue 10.15-13.30, Thu 10.15-11.45 (The whole course will be taught online via Google Meet).


Useful links and stuff

Note: you can download the slides of the course by clicking on the "PROGRAM" of the subject (see below).

1) UCI Machine Learning Repository of benchmark datasets of classification and regression problems, including documentation and bibliography.

2) Original neural network software simulator for Linux (NeuroSimulator.tgz). Save the file in a directory, enter the directory, and type "tar -xvzf NeuroSimulator.tgz".

3) Experimental environment for the IRIS classification task (Anderson/Fisher) (3 classes, 4 features reduced to 2, linear discriminant and MLP, Linux o.s.). Save the file in a directory, enter the directory, type "tar -xvzf iris.tgz", finally type "more README.txt" for instructions. The graphical plots are meant to be realized via "gnuplot".


PROGRAM (TENTATIVE)

Note 1: as it happens, part of the slides may be posted/updated in the proceedings of the course.

Note 2: the items in violet are not required for the exams of the students enrolled in the Applied Mathematics track.

1. SUB-SYMBOLIC AI

1.1 Introduction
Decision-making in machines. Classification problems. Feature extraction and classifiers. Feature kinds: numeric (discrete/continuous), symbolic and qualitative. Discriminant function. Intuitive approaches: separation line, separation surface, statistical characterization of the distribution. Classification over static/dynamic data. Review of the fundamentals of multivariate statistics. The Normal distribution..

1.2 Supervised Learning

1.2.1 Bayes decision theory
Bayes' Theorem. Bayesian risk, probability of error. Classification: 2-class vs c-class. Separation surfaces. Discriminant functions: the Normal case: template matching, min. distance classifiers, linear discriminant.

1.2.2 Parametric Estimators
Notion of parametric estimate in the supervised case. Maximum likelihood (ML) estimation. ML estimation for the parameters of the Normal pdf. Demo of a parametric Bayes classifier. Fundamentals of validation and model selection.

1.2.3 Nonparametric Estimators
Fundamentals of density estimation. Parzen Window and Kn-nearest neighbor. Nearest neighbor (NN) and k-NN algorithms.

1.2.4 Artificial neural networks (ANN)
Definitions; MLP and backpropagation; Demo of MLPs for regression; radial basis functions (RBF); Cybenko's universal property; probabilistic interpretation of ANNs; relation between MLP and Bayesian classifiers; fundamentals of generalization and regularization; mixture of experts; autoassociative ANNs.

1.3 Unsupervised learning
Mixture density, identifiability, unsupervised ML estimates. Special case: ML for mixtures of Gaussian components, k-means. Clustering: similarity measures, iterative optimization, partitioning clustering, hierarchical clustering (agglomerative, divisive). Competitive ANNs and k-means; ANN with ML for density estimation. Parzen Neural Networks. Density estimates via soft-constrained ANNs that satisfy Kolmogorov axioms of probability.

2. SYMBOLIC AI

2.1 Problem Solving
Introduction. Uninformed search (BFS, DFS, uniform cost search, iterative deepening). Informed search (heuristics, optimality, completeness, search algorithms with heuristic, A*).

2.2 Constraint Satisfaction
Introduction. Local search. Applications to game playing (8-queens). Main reparative algorithms (Simulated Annealing, Hill Climbing, Local Beam Search, Genetic algorithms).


        Based on Dream Template