LIN 105: Language Learning in Humans and Machines

Instructor	Class Day/Time	Location	Email	Office Hours	Office
Masoud Jasbi	Tue + Thu 12:10-1:30	GIEDT 1006	jasbi@ucdavis.edu	Tue 2-3	Kerr 279

Schedule

Week	Month	Date	Topic	Content	Videos	Readings
1	Apr	2	The Basic Problem(s) of Language Learning	Formal vs. Functional Approaches to Language, Modularity vs. Interactivity, Learnability and Poverty of the Stimulus, The Segmentation Problem, The Invariance Problem, The Form-Meaning Mapping Problem		Alishahi (2011). Ch. 1: Overview Clark (2016) Ch. 1: Acquiring Language
		4
2		9	The Status of the Stimulus: Poor? Rich? Both? Neither?	The Poverty of the Stumulus Argument, Input Langauge Quantity and Quality, Properties of Child-Directed Speech, Learning from Overheard Speech		Pearl (2021): Poverty of the Stimulus without Tears Clark (2016) Ch.2: In Conversation with Children Bunce et al: Cross-cultural Examination of Children's Language Experience
		11
3		16	Model Basics: Symbolic and Bayesian Models	Marr's Levels of Cognitive Modeling, Cognitive Plausibility, Symbolic Rule-Based Models, Bayesian Learning	1. Bayes Theorem, The Geometry of Changing Beliefs 2. The Quick Proof of Bayes Theorem	Alishahi (2011). Ch. 2: Computational Models of Language Learning Gibson and Wexler (1994):Triggers McElreath (2023): Statistical Rethinking Ch. 2: Small Worlds and Large Worlds Griffiths (2009): Connecting human and machine learning via probabilistic models of cognition Jurafsky and Martin (2023): N-gram Language Models
		18
4		23	Connectionism and Neural Networks	Binary Classification with Logistic Regression, Multinomial Logistic Regression, Learning in Logistic Regression, Loss Function, Gradient Descent, McCulloch-Pitts Neuron, Feed Forward Neural Networks, Back Propagation of Error	1. What is a Neural Network? 2. Gradient Descent, how neural networks learn 3. What is backpropagation really doing? 4. Backpropagation Calculus	Jurafsky and Martin (2023) Ch. 5: Logistic Regression Jurafsky and Martin (2023) Ch.7: Neural Networks and Neural Language Models
		25
5		30	Learning Morphology	English Past Tense Verbal Morphology in Child Language, Rule-based Models of Past Tense Morphology, NN Models of Past Tense Morphology		Alishahi (2011). Ch. 4.1 and 4.2 Clark (2016) Ch.8: Modulating Word Meaning Rumelhart and McClelland (1985): On Learning the Past Tenses of English Verbs Pinker and Ullman + McClelland and Patterson (2002): The past and future of past tense Optional: Kirov and Cotterell (2018): Recurrent Neural Networks in Linguistic Theory Optional: Corkery et al (2019)'s response to Kirov and Cotterell (2018)
	May	2
6		7	Word Learning	Form-Meaning Mapping Problem, Referntial Learning, Associative Learning, Cross-situational Learning,		Alishahi (2011). Ch. 3: Word Learning Siskind (1996): A computational study of word-to-meaning mappings Frank, Goodman, Tenenbaum (2009): Using Speakers' Referential Intentions to Model Early Cross-Situational Word Learning
		9
7		14	Model Basics: Recurrent Neural Networks	Elman Networks, RNNs as Language Models, Stacked and Bidirectional RNNs, Long Short-Term Memory Models, Attention		Jurafsky and Martin (2023) Ch.9: RNNs and LSTMs
		16
8		21	Learning Syntax	Representing Time in a Network, Discovering the Notion of "word", Learning Basic Sentence Structure with Simple RNNs, Bayesian Context-Free Grammar Induction		Alishahi (2011). Ch. 4 Elman (1990): Finding Structure in Time Perfors, et al (2011): The Learnability of Abstract Syntactic Principles Huebner et al (2021): BabyBERTa
		23
9		28	Model Basics: Transformers and Large Language Models	Transformer Architecture, Self-Attention, Multihead Attention, Transformer Blocks, Embeddings, Large Language Models, Bidirectional Transformer Encoders, Contextual Embeddings, Fine-Tuning	1. Visual Intro to Transformers 2. Attention in Transformers	Jurafsky and Martin (2023) Ch.10: Transformers and Large Language Models Jurafsky and Martin (2023) Ch. 11: Fine-tuning and Masked Language Models
		30
10	Jun	4	Word Learning in LLMs	Grounded Word Learning with Multimodal Models, Word Learning in Children vs. LLMs		Vong et al (2024): Grounded language acquisition through the eyes and ears of a single child Chang and Bergen (2022): Word Acquisition in Neural Language Models
		6

Course Objectives

	Objective	Course Component
1	Introduce the foundations of computational learning models	Readings, Lectures
2	Introduce the basic findings in children's language development	Readings, Lectures
3	Practice critical and scientific thinking and research	Assessment

Syllabus

Assessment
Research Skills	100 Points
		Reserach Question and Bibliography	10 Points	A one page bullet point presentation of the research question for your project and the papers you plan to read on it.
		Introduction of the Research Question and Literature Review	20 Points	A two page document presenting the research question and summarizing prior literature on it.
		Research Proposal	30 Points	A two page document presenting the research question, summarizing prior literautre, and proposing methods for your study and the results you expect.
		Final Presentation	40 Points	A 7 minute flash-talk presentation of your resarch question, prior literature, proposed methods for your study and results you expect followed with a 7 minute Q and A.
Policies
Groups		You can do your research assignment individually or form groups with other students. Group submissions must also submit a separate page explaining author contributions. You can see an example of author contribution in scientific journal by clicking on this link. You should also mention whether authors have had roughly equal contribution in your judgment and the grade should be distributed equally or not. Common examples of unequal contribution are members missing meetings and work sessions of the group frequently or being completely absent and unresponsive until deadlines or even after. Disputes will be resolved on a case by case basis.
Late Submission		Late assignments will be graded as though they were not late, but then 5% of the grade earned will be deducted for each day the assignment is late, with a maximum penalty of 50%. All late work must be turned in by the Friday before your final exam. This policy can be waived if lateness is due to medical reasons or other special circumstances.
Submission Format		Submit your assignments using Canvas. Files should be in PDF. Typed assignments should use Times New Roman (12pt), 1 inch margins, 1.5 line spacing. Do not include your name or any identifying information in the assignment. In order to avoid grading biases, assignments are graded anonymously.
Grading		We use the following grading scale: A+ = 100-97 A = 97-93, A- = 93-90, B+ = 90-87, B = 87-83, B- = 83-80, C+ = 80-77, C = 77-73, C- = 73-70, D+ = 70-67, D = 67-63, D- = 63-60, F = 60-0. For any submission, if you believe there have been grading mistakes, you can ask for re-grading. The assignment will be graded by a new grader and the second grade will be recorded.
Integrity		We follow the UC Davis code of academic conduct. You are permitted to work together on the assignments. However, you must write up and submit your own unique assignments.
Accessibility		Students who may need an academic accommodation based on the impact of a disability must initiate the request with the UC Davis Student Disability Center. Professional staff will evaluate the request, recommend reasonable accommodations, and prepare a letter of accommodation for the faculty. Students should contact the SDC as soon as possible since timely notice is needed to coordinate accommodations.
Addressing the Instructor		I prefer Masoud and he/his/him for pronouns. No titles or last name needed.
Philosophy
Participation		We believe that our class benefits enormously from you sharing your thoughts and questions. Your background, life experiences, knowledge, thoughts, and ideas make you unique, and our classroom diverse. This diversity of perspectives is the foundation of learning in a classroom. At a larger scale and within a scientific community, it is also a major contributor to scientific progress. Therefore, sharing your thoughts and questions can help us learn and build a wider, stronger community of scholars. Some of you may worry that your classmate's asking questions and sharing ideas may disrupt the class progress. Judging when to ask a question or share an idea is tricky but also part of education. Instead of discouraging it, we would like to practice it together. Here is flowchart that you might find useful. Ultimately, we trust your judgments.
Questions		We genuinly believe that there are no "stupid" questions in a classroom. The point of going to a class is to learn together and questions are our best tool to achieve that. It is easy to show that your question will help us learn no matter what. Your question is either: (1) not framed well; in which case you give us a chance to explain the topic better. Chances are we did not explain it well the first time and many of your classmates are wondering about it too. (2) framed well and has an answer we know; in which case we can help you as well as your classmates who have the same question learn it too! You have also helped us consolidate our knowledge by explaining it again. (3) framed well but has an answer we do not know; in which case we can find the answer together and your question has helped all of us learn! (4) framed well and does not have an answer yet; in which case you found a research topic someone can start working on and benefit the field! As you see, your question has helped our learning either way. So please ask!