來自哈佛大學Boaz Barak教授最新報告《Empirical challenges to theories of deep learning》
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.
近年來,圖表示學習的研究激增,包括深度圖嵌入(deep graph embeddings)技術、卷積神經網絡對圖結構數據的泛化以及受置信傳播啟發的神經信息傳遞方法。
與此同時,圖表示學習的這些進步促成了許多領域的最新成果,包括化學合成、3D 視覺、推薦系統、問題解答和社交網絡分析等。
加拿大麥吉爾大學計算機科學助理教授 William Hamilton 的《圖表示學習》(Graph Representation Learning)報告系統性介紹最新圖表示學習的進展。
隨著機器學習模型越來越多地用于在醫療保健和刑事司法等高風險環境中幫助決策者,確保決策者(最終用戶)正確理解并因此信任這些模型的功能是很重要的。本報告旨在讓學生熟悉可解釋和可解釋ML這一新興領域的最新進展。在本報告中,我們將回顧該領域的重要論文,理解模型可解釋和可解釋的概念,詳細討論不同類別的可解釋模型(如基于原型的方法、稀疏線性模型、基于規則的技術、廣義可加性模型),事后解釋(黑箱解釋包括反事實解釋和顯著性圖),并探索可解釋性與因果關系、調試和公平性之間的聯系。該課程還將強調各種應用,可以極大地受益于模型的可解釋性,包括刑事司法和醫療保健。
【導讀】來自Fariz Darari博士的一份簡明《神經網絡與深度學習》的講義,64頁ppt,可以學習。
【導讀】2020新年伊始,多倫多大學Amir-massoud Farahmand和Emad A. M. Andrews博士開設了機器學習導論課程,介紹了機器學習的主要概念和思想,并概述了許多常用的機器學習算法。它還可以作為更高級的ML課程的基礎。
課程地址:
//amfarahmand.github.io/csc311/
機器學習(ML)是一組技術,它允許計算機從數據和經驗中學習,而不需要人工指定所需的行為。ML在人工智能作為一個學術領域和工業領域都變得越來越重要。本課程介紹了機器學習的主要概念和思想,并概述了許多常用的機器學習算法。它還可以作為更高級的ML課程的基礎。
本課程結束時,學生將學習(大致分類)
機器學習問題:監督(回歸和分類),非監督(聚類,降維),強化學習
模型:線性和非線性(基擴展和神經網絡)
損失函數:平方損失、交叉熵、鉸鏈、指數等。
Regularizers: l1和l2
概率觀點:最大似然估計,最大后驗,貝葉斯推理
偏差和方差的權衡
集成方法:Bagging 和 Boosting
ML中的優化技術: 梯度下降法和隨機梯度下降法
課程目錄:
參考資料:
(ESL) Trevor Hastie, Robert Tibshirani, and Jerome Friedman, The Elements of Statistical Learning, 2009.
(PRML) Christopher M. Bishop, Pattern Recognition and Machine Learning, 2006.
(RL) Richard S. Sutton and Andrew G. Barto Reinforcement Learning: An Introduction, 2018.
(DL) Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016), Deep Learning
(MLPP) Kevin P. Murphy, Machine Learning: A Probabilistic Perspective, 2013.
(ISL) Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani, Introduction to Statistical Learning, 2017.
() Shai Shalev-Shwartz and Shai Ben-David Understanding Machine Learning: From Theory to Algorithms, 2014.
(ITIL) David MacKay, Information Theory, Inference, and Learning Algorithms, 2003.
When I started out, I had a strong quantitative background (chemical engineering undergrad, was taking PhD courses in chemical engineering) and some functional skills in programming. From there, I first dove deep into one type of machine learning (Gaussian processes) along with general ML practice (how to set up ML experiments in order to evaluate your models) because that was what I needed for my project. I learned mostly online and by reading papers, but I also took one class on data analysis for biologists that wasn’t ML-focused but did cover programming and statistical thinking. Later, I took a linear algebra class, an ML survey class, and an advanced topics class on structured learning at Caltech. Those helped me obtain a broad knowledge of ML, and then I’ve gained deeper understandings of some subfields that interest me or are especially relevant by reading papers closely (chasing down references and anything I don’t understand and/or implementing the core algorithms myself).
機器學習可解釋性,Interpretability and Explainability in Machine Learning