APL for Data Science: Array Programming Tutorial With Jupyter and Python
In a hands-on session at the Data Science Innovators Conference 2023, Ontario Tech University students Benjamin Fedoruk and Max Sun introduced APL for data science. They covered array operators, right-to-left evaluation, tacit functions, and a Jupyter and Python demo that plotted social media emotion data.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Everything is an array. APL operators work element by element on arrays, with a monadic and a dyadic form for each symbol.
- Right to left, not PEMDAS. Without brackets APL evaluates from right to left, which changes results.
- Short functions. A mean is plus-slash divided by tally, written as a dfn or as a tacit train.
- Plays with Python. Dyalog offers Jupyter notebooks and a Python interface, per Sun's demo.
- Expect trial and error. Sun said interactive debugging is how APL is done in practice.
In a hands-on session at the Data Science Innovators Conference 2023, two Ontario Tech University students introduced APL, the array-oriented programming language, to data scientists used to Python. Benjamin Fedoruk, an undergraduate studying applied and industrial mathematics, and Max Sun, who said he works part-time as a junior APL developer at BCA Research, ran the session from a shared room in Oshawa, Ontario. Fedoruk covered the language basics in a browser; Sun then showed APL working alongside Python and Jupyter notebooks to explore and plot data.
What is APL and why does it look so different?
Fedoruk described APL as a different paradigm from the object-oriented and functional styles most programmers know. It is array-oriented, was invented in the 1960s, and began as a "tool for thought," a concise notation for mathematical ideas that later evolved into a programming language for data science and visualization.
He acknowledged that the syntax is terse and that many people call it hard to read. He argued that every language looks unreadable to a newcomer, citing Java and C, and that APL becomes readable with experience. Attendees could follow along in the browser at tryapl.org, run by Dyalog, one flavor of APL, with no download.
How do operators and arrays work?
In APL, Fedoruk said, everything is an array, including single numbers. Each symbol has two modes: dyadic, taking two inputs, and monadic, taking one. Division, for instance, divides two numbers in its dyadic form and returns the reciprocal in its monadic form. Operations apply element by element, so adding the arrays 1 2 3 and 4 5 6 gives 5 7 9.
He walked through a symbol set that covers arithmetic, powers, logarithms, combinations, maximum and minimum, residue (modulus), set operations, trigonometry, tally (length) and matrix inversion. The iota symbol generates a range and counts from one by default, which he showed adjusting to start from zero.
What trips up newcomers?
Fedoruk said APL does not follow the usual order of operations. It evaluates right to left unless brackets say otherwise. He demonstrated that two plus three times five gives 17, while five times three plus two gives 25, because the addition runs first.
The reduce form of the slash puts a function between every element of a list, so plus-slash sums a list. His quiz exposed a trap: star-slash over one to five is not a product, because star means exponentiation in APL. Sun later explained why that expression returned a domain error: it raises one to the power of a tower of exponents that grows enormous by the fifth term.
How do you build a mean in APL?
Fedoruk built the mean of a list two ways. The sum is plus-slash, the count is tally, and dividing them gives the mean. He then wrapped that logic as a "dfn," a function in curly brackets that uses omega for its single argument, and finally as a tacit, or point-free, function using a three-function train, or fork. He called trains a notoriously difficult concept and left an exercise: split a non-negative number into its integer and fractional parts. He also showed the "selfie" operator, which swaps the two arguments of a function.
How does APL work with Python and Jupyter?
Sun showed that Dyalog provides Jupyter notebooks with an APL kernel, which he described as a little tricky to install, and a Python interface that lets APL run Python code. In his demo he loaded a pickled data set of social media emotion scores using pandas, saved it to CSV, read it into APL and indexed columns, noting that APL's indexing notation predates NumPy and MATLAB.
He plotted a vector of more than 10,000 entries with a built-in plot function and drew histograms of emotion scores. Happy scores mostly fell between zero and 0.1, he said, while sad scores showed more than a thousand entries at the top of the scale. He then used SharpPlot, a charting library that ships with Dyalog, and said the sometimes messy debugging was "how you do APL with data science in practice."
Who are the presenters?
Fedoruk said he is founding president of the Northern Shores Innovation Institute, which gives young people access to research and development opportunities. Sun described himself as interested in functional programming and type theory. Both said they were presenting during university exam season.
What is next from AI Connect?
AI Connect continues to run practitioner workshops and webinars. Its 2027 schedule includes 24 free live webinars, listed on the webinars page, and COLLIDE 2027 takes place Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center in Atlanta; see COLLIDE. Registration is coming soon.
Frequently asked questions
What is APL?
Fedoruk described it as an array-oriented language invented in the 1960s as a concise notation for thought, since evolved into a language for data science and visualization.
How can I try APL without installing anything?
He pointed attendees to tryapl.org, a Dyalog-run site that runs APL in the browser.
Why does APL give different answers from other languages?
It evaluates right to left unless brackets are used, so five times three plus two equals 25.
Can APL work with Python and Jupyter?
Sun demonstrated Dyalog's Jupyter kernel and a Python interface, loading pandas data and plotting it from APL.
This article is based on remarks by Benjamin Fedoruk, Researcher, Ontario Tech University; Max Sun, Ontario Tech University at Data Science Innovators Conference 2023 on April 12, 2023. More coverage: More AI Connect Events.
Filed under: AI Use Cases & Applications, Data & AI Infrastructure
