Click to run and edit this dialog:

Run in SolveIt

BPL Working with Lists

!pip install -U basedpl
Requirement already satisfied: basedpl in /app/data/.local/lib/python3.12/site-packages (0.1.75)
Requirement already satisfied: fastcore in /usr/local/lib/python3.12/site-packages (from basedpl) (2.1.16)
%load_ext basedpl.notebooks
import basedpl
basedpl.__version__
'0.1.75'

Working with a list of numbers

In this dialog we take one small set of data, a week of temperatures, and ask it everyday questions: which day was hottest? How many warm days were there? Which days were they? Each question introduces one or two new glyphs.

Glyphs we'll be using:

GlyphPlain English
≢count the items
⌈ ⌊the bigger / smaller of two numbers
\running results (running total, and so on)
⌽reverse
↑ ↓take / drop items from the start or end
> <greater than / less than
#keep only some items (filter)
⍸where: the positions of the true answers
.[ ]pick items by position
⍋ ⍒sort order: smallest first / largest first
∊is it in the list?
∪remove duplicates
⌸group matching items together
⊢right: give back the right-hand side, ignore the left
+/÷≢a train: functions combined into a new function, without naming the data

The data

A list (BPL calls it a vector) is numbers written next to each other with spaces in between. Text goes in double quotes, so "Mon" is a piece of text, and "Mon" "Tue" is a list of two pieces of text.

temps holds the midday temperature (°C) for each day of one week, and days holds the day names in the same order. So item 0 of temps (12) belongs to item 0 of days ("Mon"), and so on. Remember that BPL counts positions from 0.

%%bpl
temps←12 15 11 18 20 14 9
days←"Mon" "Tue" "Wed" "Thu" "Fri" "Sat" "Sun"
temps
days
12 15 11 18 20 14 9
┌→──────────────────────────────────────────┐
│ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ │
│ │Mon│ │Tue│ │Wed│ │Thu│ │Fri│ │Sat│ │Sun│ │
│ └───┘ └───┘ └───┘ └───┘ └───┘ └───┘ └───┘ │
└∊──────────────────────────────────────────┘

A note on the printout above: each piece of text prints inside its own small box, and the big box around them shows that they all belong to one list.

1. Count the items: ≢

≢ (called tally) counts how many items a list has.

%%bpl
≢temps   ⍝ how many temperatures?
≢days    ⍝ how many day names?
7ₓ
7ₓ

Both have 7 items, one for each day.

2. Total and average

+/ adds up a list. ÷ is division. The average is the total divided by the count.

%%bpl
total←+/temps     ⍝ add up all 7 temperatures
total
total÷≢temps      ⍝ average = total ÷ count
99
14.14285714

The average temperature was about 14.1 °C.

A shorter way to write the average: a train

Above, we wrote the average as total÷≢temps, mentioning temps each time. BPL also lets you build a new function out of other functions, without mentioning the data at all. This is called a train.

When three functions sit side by side, f g h, BPL reads (f g h) x as (f x) g (h x):

  • apply the left function to x,
  • apply the right function to x,
  • combine the two results with the middle function.

So +/÷≢ means (+/x) ÷ (≢x): the total divided by the count, which is the average.

%%bpl
mean←+/÷≢       ⍝ a train: total ÷ count
mean temps      ⍝ average temperature
mean 1 2 3 4    ⍝ works on any list
14.14285714
2.5

mean temps gives the same 14.14 as before. Because mean doesn't mention temps, it works on any list, and we'll reuse it later in this dialog.

3. The bigger or smaller of two numbers: ⌈ ⌊

With a number on each side:

  • ⌈ gives the bigger of the two (think of it pointing up).
  • ⌊ gives the smaller of the two (pointing down).
%%bpl
3⌈7   ⍝ bigger of 3 and 7
3⌊7   ⍝ smaller of 3 and 7
7
3

4. Hottest and coldest day: ⌈/ ⌊/

/ puts a function between every item of a list. So +/ adds them all up, and ⌈/ compares them all and keeps the biggest:

⌈/12 15 11 means 12⌈15⌈11, which is 15.

%%bpl
⌈/temps   ⍝ hottest temperature
⌊/temps   ⍝ coldest temperature
20
9

The hottest temperature was 20 and the coldest was 9.

5. Running results: \

\ (called scan) works like /, but keeps every step along the way instead of only the final answer.

  • +\temps gives a running total: each item is the sum of all temperatures up to that day.
  • ⌈\temps gives the hottest so far: each item is the highest temperature up to that day.
%%bpl
temps
+\temps   ⍝ running total
⌈\temps   ⍝ hottest so far
12 15 11 18 20 14 9
12 27 38 56 76 90 99
12 15 15 18 20 20 20

Read across: by Wednesday the running total is 38 (12 + 15 + 11), and the hottest so far is 15 (Tuesday's).

6. Reverse: ⌽

⌽ turns a list back to front.

%%bpl
temps
⌽temps
⌽days
12 15 11 18 20 14 9
9 14 20 18 11 15 12
┌→──────────────────────────────────────────┐
│ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ │
│ │Sun│ │Sat│ │Fri│ │Thu│ │Wed│ │Tue│ │Mon│ │
│ └───┘ └───┘ └───┘ └───┘ └───┘ └───┘ └───┘ │
└∊──────────────────────────────────────────┘

7. Take and drop: ↑ ↓

With a number on the left and a list on the right:

  • 3↑temps takes the first 3 items.
  • 2↓temps drops the first 2 items and keeps the rest.
  • A negative number counts from the end, so ¯2↑temps takes the last 2 items. (¯ is BPL's minus sign for negative numbers.)
%%bpl
temps
3↑temps    ⍝ first 3 days (Mon, Tue, Wed)
2↓temps    ⍝ everything except the first 2 days
¯2↑temps   ⍝ last 2 days: the weekend
¯2↑days
12 15 11 18 20 14 9
12 15 11
11 18 20 14 9
14 9
┌→────────────┐
│ ┌→──┐ ┌→──┐ │
│ │Sat│ │Sun│ │
│ └───┘ └───┘ │
└∊────────────┘

8. Ask a yes/no question of every item: > <

temps>14 asks "is this temperature greater than 14?" for every day at once. < asks "less than?".

The answers are Booleans: $t means true and $f means false. When you add Booleans, each true counts as 1, so +/ counts how many answers are true.

%%bpl
temps>14       ⍝ was each day warmer than 14?
+/temps>14     ⍝ how many days were warmer than 14?
$f $t $f $t $t $f $f
3ₓ

Three days were warmer than 14: the true answers are at Tuesday, Thursday and Friday.

9. Keep only some items: #

# filters a list. Put a list of yes/no answers on the left: it keeps the items where the answer is yes and drops the rest.

First a small example: 1 0 1#10 20 30 keeps the 1st and 3rd items.

(If you know APL: APL uses / for this. BPL uses # and keeps / for reduce only.)

%%bpl
1 0 1#10 20 30   ⍝ keep, drop, keep
10 30

Now use the answers from step 8 to keep the warm days. Storing the answers as hot lets us filter both lists with the same yes/no answers.

%%bpl
hot←temps>14   ⍝ yes/no answer for each day
0+hot          ⍝ 0+ coerces Booleans into numeric values
hot#temps      ⍝ the warm temperatures
hot#days       ⍝ the warm days' names
0 1 0 1 1 0 0
15 18 20
┌→──────────────────┐
│ ┌→──┐ ┌→──┐ ┌→──┐ │
│ │Tue│ │Thu│ │Fri│ │
│ └───┘ └───┘ └───┘ │
└∊──────────────────┘

10. Where: ⍸

Sometimes you want the positions of the true answers, not the items themselves. ⍸ (where) gives those positions. Remember that positions count from 0, so Monday is 0.

%%bpl
0+temps>14
⍸temps>14   ⍝ positions of the warm days
0 1 0 1 1 0 0
[1 3 4]ₓ

Positions 1, 3 and 4 are Tuesday, Thursday and Friday.

11. Pick items by position: .[ ]

days.[i] picks the items of days at the positions held in i. It's easiest to store the positions in a name first, then put that name in the brackets.

%%bpl
i←⍸temps>14   ⍝ positions of the warm days
i
days.[i]      ⍝ the names at those positions
[1 3 4]ₓ
┌→──────────────────┐
│ ┌→──┐ ┌→──┐ ┌→──┐ │
│ │Tue│ │Thu│ │Fri│ │
│ └───┘ └───┘ └───┘ │
└∊──────────────────┘

This gives the same names as step 9, found a different way. Picking by position becomes useful for sorting, which comes next.

12. Sorting: ⍋ ⍒

⍋ (grade up) does not give you the sorted numbers. It gives the positions of the items, in order from smallest to largest.

Small example with x←30 10 20:

  • The smallest item, 10, is at position 1.
  • Next is 20, at position 2.
  • The largest, 30, is at position 0.

So ⍋x is 1 2 0. To get the sorted numbers, pick the items at those positions with x.[⍋x] (step 11).

⍒ (grade down) works the same way but gives largest first.

%%bpl
x←30 10 20
⍋x        ⍝ positions, smallest first
x.[⍋x]    ⍝ the items at those positions: sorted
⍒x        ⍝ positions, largest first
x.[⍒x]    ⍝ sorted, largest first
[1 2 0]ₓ
10 20 30
[0 2 1]ₓ
30 20 10

Why return positions rather than sorted numbers? Because the same positions can reorder another list. Here we sort the days from hottest to coldest by using the positions from temps to pick from days.

%%bpl
i←⍒temps   ⍝ positions, hottest first
temps.[i]  ⍝ temperatures, hottest first
days.[i]   ⍝ the matching day names
20 18 15 14 12 11 9
┌→──────────────────────────────────────────┐
│ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ ┌→──┐ │
│ │Fri│ │Thu│ │Tue│ │Sat│ │Mon│ │Wed│ │Sun│ │
│ └───┘ └───┘ └───┘ └───┘ └───┘ └───┘ └───┘ │
└∊──────────────────────────────────────────┘

Friday was the hottest day and Sunday the coldest.

13. Is it in the list? ∊

a∊b asks, for each item of a, "does this appear anywhere in b?". The answer is one true/false per item of a.

Small example: which of 2, 5 and 9 are in the list 1 2 3 4 5?

%%bpl
2 5 9∊1 2 3 4 5   ⍝ 2 yes, 5 yes, 9 no
15 18 99∊temps    ⍝ did we ever see 15, 18 or 99?
$t $t $f
$t $t $f

14. Remove duplicates: ∪

∪ (unique) keeps the first appearance of each different item and removes the repeats.

%%bpl
∪3 1 3 2 1   ⍝ each number once, in order of first appearance
3 1 2

Now add one more list: the weather on each day. Like days, it is in the same order as temps.

%%bpl
w←"sun" "rain" "sun" "cloud" "sun" "rain" "sun"   ⍝ Mon to Sun
∪w   ⍝ the different kinds of weather
┌→─────────────────────┐
│ ┌→──┐ ┌→───┐ ┌→────┐ │
│ │sun│ │rain│ │cloud│ │
│ └───┘ └────┘ └─────┘ │
└∊─────────────────────┘

15. Group and count: ⌸

⌸ (key) groups matching items together and runs a function on each group.

In {≢⍵}⌸w:

  • ⌸ finds each different value in w ("sun", "rain", "cloud", in order of first appearance, the same order as ∪w).
  • For each value, ⍵ is the list of positions where it appears.
  • ≢⍵ counts those positions.

The result is how many times each kind of weather happened.

%%bpl
{≢ ⍵}⌸w            ⍝ count of each weather type
[∪w ⋄ {≢⍵}⌸w]      ⍝ the same counts, with the names above them
[4 2 1]ₓ
┌→─────────────────────┐
↓ ┌→──┐ ┌→───┐ ┌→────┐ │
│ │sun│ │rain│ │cloud│ │
│ └───┘ └────┘ └─────┘ │
│    4ₓ     2ₓ      1ₓ │
└∊─────────────────────┘

The second line uses [ ⋄ ] to stack two rows into a table: the names on top and the counts underneath. Sun appeared 4 times, rain twice and cloud once.

16. Group one list by another: ⌸ with a left side

Put a list on the left of ⌸ to group a different list by it. In w{...}⌸temps:

  • the temperatures are grouped by the weather on that day;
  • for each weather type, ⍵ is the list of temperatures on those days.

So w{(+/⍵)÷≢⍵}⌸temps gives the average temperature for each kind of weather. (Write it without spaces: in BPL, spaces split a line into separate pieces.)

%%bpl
w
w{(+/⍵)÷≢⍵}⌸temps             ⍝ average temperature per weather type
[∪w ⋄ w{(+/⍵)÷≢⍵}⌸temps]       ⍝ with the names above
┌→──────────────────────────────────────────────┐
│ ┌→──┐ ┌→───┐ ┌→──┐ ┌→────┐ ┌→──┐ ┌→───┐ ┌→──┐ │
│ │sun│ │rain│ │sun│ │cloud│ │sun│ │rain│ │sun│ │
│ └───┘ └────┘ └───┘ └─────┘ └───┘ └────┘ └───┘ │
└∊──────────────────────────────────────────────┘
13 14.5 18
┌→─────────────────────┐
↓ ┌→──┐ ┌→───┐ ┌→────┐ │
│ │sun│ │rain│ │cloud│ │
│ └───┘ └────┘ └─────┘ │
│    13   14.5      18 │
└∊─────────────────────┘

Sunny days averaged 13 °C, rainy days 14.5 °C, and the one cloudy day was 18 °C.

17. Grouping with a train instead of braces

In step 16 we wrote the average inside braces: {(+/⍵)÷≢⍵}. We already made mean back in step 2, so can we hand that to ⌸ instead? Yes, but first we need one more small glyph.

⊢ (right) gives back whatever is on its right, and ignores anything on its left.

%%bpl
5⊢1 2 3    ⍝ ignores the 5
⊢1 2 3     ⍝ nothing on the left: just gives 1 2 3
1 2 3
1 2 3

Why ⌸ needs it. ⌸ calls its function with two things:

  • on the left, the group's key (such as "sun");
  • on the right, the group's values (the temperatures on sunny days).

A braces function only uses ⍵ (the right side), so it ignores the key automatically. A train like mean would try to use both sides, which isn't what we want.

mean⊢ fixes this. Two functions side by side, f g, mean "do g, then do f to its result". So key (mean⊢) values first does key⊢values, which keeps just the values, and then takes their mean.

%%bpl
w(mean⊢)⌸temps            ⍝ mean of each group; the key is ignored
[∪w ⋄ w(mean⊢)⌸temps]     ⍝ with the names above
13 14.5 18
┌→─────────────────────┐
↓ ┌→──┐ ┌→───┐ ┌→────┐ │
│ │sun│ │rain│ │cloud│ │
│ └───┘ └────┘ └─────┘ │
│    13   14.5      18 │
└∊─────────────────────┘

Same answers as step 16, with no braces and no ⍵.

Probe the inputs passed to the grouped function

We can temporarily replace mean with {⍵}, a function that simply returns its right argument. This lets us inspect the temperature list that ⌸ passes to the function for each weather group.

%%bpl
w
temps
w{⍵}⌸temps   ⍝ reveal the right argument for each weather group
┌→──────────────────────────────────────────────┐
│ ┌→──┐ ┌→───┐ ┌→──┐ ┌→────┐ ┌→──┐ ┌→───┐ ┌→──┐ │
│ │sun│ │rain│ │sun│ │cloud│ │sun│ │rain│ │sun│ │
│ └───┘ └────┘ └───┘ └─────┘ └───┘ └────┘ └───┘ │
└∊──────────────────────────────────────────────┘
12 15 11 18 20 14 9
┌→─────────┐
↓12 11 20 9│
│15 14  0 0│
│18  0  0 0│
└~─────────┘

The result should show three groups, in the same order as ∪w:

  • sun: 12 11 20 9
  • rain: 15 14
  • cloud: 18

Now we can isolate one representative call. The key ("sun") is on the left and that group's temperatures are on the right.

%%bpl
"sun"⊢12 11 20 9          ⍝ ⊢ discards the key and keeps the values
mean("sun"⊢12 11 20 9)    ⍝ explicitly discard, then take the mean
"sun"(mean⊢)12 11 20 9    ⍝ the two-function train does the same thing
12 11 20 9
13
13

All three lines expose the same path through the calculation: ⊢ keeps only the right-side values, and mean then averages them. The last two lines should both produce 13, confirming that mean⊢ ignores the group key.

Building ⊢ into the function. You can also put the ⊢ inside the definition. That gives a function that always means "the mean of whatever is on the right". We'll call it meanR (R for right), so it doesn't replace mean.

  • It still works on its own: meanR temps.
  • With ⌸, it needs no extra ⊢.

The last line is written without spaces, because in BPL a space would split it into separate pieces. The brackets around w keep the two names w and meanR apart; without them, wmeanR would read as one name.

%%bpl
meanR←(+/÷≢)⊢       ⍝ mean of the right-hand side
meanR temps         ⍝ on its own
(w)meanR⌸temps      ⍝ grouped by weather
14.14285714
13 14.5 18

18. Faster counting and totals

The same ⊢ idea simplifies the counts from step 15: (≢⊢)⌸w counts each group. And w(+/⊢)⌸temps adds up the temperatures in each group.

These are also faster. BPL recognises a few patterns, such as (≢⊢)⌸, (+/⊢)⌸ and (⌈/⊢)⌸, and works them out in a single pass over the data, without running a function once for every group. With 7 days you won't notice, but with millions of rows you would.

≡ (match) checks that each new version gives exactly the same result as the braces version.

%%bpl
(≢⊢)⌸w                               ⍝ count of each weather type
({≢⍵}⌸w)≡(≢⊢)⌸w                     ⍝ same as step 15?
w(+/⊢)⌸temps                         ⍝ total temperature per weather type
(w{+/⍵}⌸temps)≡w(+/⊢)⌸temps         ⍝ same as the braces version?
[4 2 1]ₓ
$t
52 29 18
$t

Both checks give $t (true): the train versions give the same answers as the braces versions.

Your turn

  1. How many days were colder than 12? (Use < and +/.)
  2. Which days were they? Find their names in two ways: with #, and with ⍸ and .[ ].
  3. List the temperatures from coldest to hottest. (Use ⍋.)
  4. What was the hottest temperature for each kind of weather? (Use ⌸ with w on the left, and ⌈/ inside the braces.)