Admission requirements
Elementary knowledge of data structures (hash functions, hash tables, dictionaries, graphs, sparse matrices), statistics (binomial distribution) and combinatorics (permutations, combinations). Basic programming skills in Python (with NumPy and Matplotlib packages).
Description
The traditional data mining techniques are mainly focused on solving classification, regression and clustering problems. However, the recent developments in ICT led to the emergence of new sorts of massive data sets and related data mining problems. Consequently, the field of data mining has rapidly expanded to cover new areas of research, such as:
processing huge (tera- or petabytes big) data sets,
fast searching for similar objects, such as: documents, images, songs, routes, etc., in collections of millions or billions of such objects,
clustering of massive data sets,
real-time analysis of data streams (internet traffic, sensor data, electronic transactions),
recommending items to visitors of internet shops,
analysing big (network) graphs, such as web sites, social networks, collaboration networks, etc.
During the course we will focus on these areas. We will start with introducing a powerful framework for processing massive data sets on distributed computers: Hadoop and MapReduce. Then a new, very general similarity search technique, Locality Sensitive Hashing, will be discussed, together with its applications to plagiarism detection, searching databases with fingerprints, finding clients with similar buying behaviour, etc. Next, several algorithms for real-time mining of data streams will be introduced: Bloom filters, random sampling, counting, estimating moments. Finally, some state-of-the art recommendation systems and algorithms for dimensionality reduction and data visualization will be introduced. The practical part of the course will consists of several programming assignments (in Python) and writing reports.
Course objectives
After completing the course, the students should:
have a general knowledge of the recent developments in the field of Data Mining
have detailed knowledge of selected techniques and their applications
gain some hands-on experience with several algorithms for mining complex data sets
be able to apply the acquired knowledge and skills to new problems
gain some experience with mining big data sets on a cluster computer
Timetable
The most recent timetable can be found at the students' website.
Mode of instruction
Lectures
Computer Lab
Practical assignments
Self-evaluated homework
Assessment method
The final mark is composed of (1) written exam (40%) (2) practical assignments (60%)
The teacher will inform the students how the inspection of and follow-up discussion of the exams will take place.
Reading list
A. Rajaraman, J. Leskovec, J. Ullman, Mining of Massive Datasets.
Registration
- You have to sign up for courses and exams (including retakes) in uSis. Check this link for information about how to register for courses.
Contact
Lecturer: Dr. Wojtek Kowalczyk.