Homo Ex Machina
“Attention Is All You Need,” the now-famous paper that put the “T” in ChatGPT, created the key building block for all your favorite chatbots.
Data scientist Joshua Rio-Ross, with a Master’s in theology, explores what “attention” really means in AI systems and how that relates to what we humans experience when we’re focused on something.
Rio-Ross questions whether AI can truly replicate human cognitive processes. He suggests that “attention” in AI and machine learning is often based on our own human understanding of that word. And while there might be similarities, there are also differences.
Take “attention spans” for example. Drawing on theological and philosophical perspectives, Rio-Ross discusses the differences between what “attention span” means for humans and what it means for machine learning models.
Such analogies should be approached with caution, Rio-Ross says. And more than that, our language about AI should not restrict how we think about our own human potential.