<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<font face="Helvetica, Arial, sans-serif">Hi Louis,<br>
<br>
most people will assume that<br>
</font><br>
<font face="Helvetica, Arial, sans-serif"><font face="Courier New,
Courier, monospace"><b>A←A,1 2 3</b><b><br>
</b><b><br>
</b></font>simply appends <b>1 2 3 </b>to <b>A</b>. But don't
bet your life on that:<br>
<font face="Courier New, Courier, monospace"><b><br>
</b><b> )CLEAR<br>
CLEAR WS<br>
<br>
∇Z←A B<br>
[1] ⊣⎕EX 'A' ◊ Z←42,B<br>
[2] ∇<br>
<br>
A←A,1 2 3<br>
A<br>
42 1 2 3<br>
<br>
<br>
</b></font>The optimizations that Elias and myself tried earlier
worked for the most part,<br>
but failed on pathological cases like the above. The detection of
such cases<br>
cannot, as the example above shows, be done statically (say at
function definition<br>
time) but has to be done at runtime. The question then is: will
the effort for the<br>
detection be amortized by the fewer copies or not? My guts feeling
is that it<br>
does not, but you can always construct a benchmark in which it
does.<br>
<br>
Our conclusion at that time was that the correctness of the result
is more important<br>
than speed.<br>
<br>
Best Regads,<br>
Jürgen Sauermann<br>
<br>
<br>
</font><br>
<div class="moz-cite-prefix">On 07/05/2017 02:20 AM, Louis de
Forcrand wrote:<br>
</div>
<blockquote
cite="mid:1579EEE0-EB62-44FD-BC97-***@bluewin.ch"
type="cite">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<div>On the subject of copying arrays, I was wondering, in GNU
APL, where in code such as</div>
<div><br>
</div>
<div>A <- A,1 2 3</div>
<div>B <- A,4 5 6</div>
<div>A <- A,7 8 9</div>
<div><br>
</div>
<div>(in that order) A is copied. It has to be copied in the third
line (unless some weird bookkeeping is done) as B referred to
A's previous value, but can be modified in place in the first
two lines.</div>
<div>
<div><br>
</div>
</div>
<div id="AppleMailSignature">Also</div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">1 2 3 + 3 4 5</div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">where the result of + can overwrite
one of its two arguments (the same goes for any scalar
function).</div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">What is the in place situation of GNU
APL?</div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">Thanks,</div>
<div id="AppleMailSignature">Louis</div>
<div><br>
On 4 Jul 2017, at 20:24, Juergen Sauermann <<a
moz-do-not-send="true"
href="mailto:***@t-online.de">***@t-online.de</a>>
wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
<font face="Helvetica, Arial, sans-serif">Hi Blake,<br>
<br>
The state of GNU APL is, I believe, this:<br>
<br>
I am not aware of any unnecessary copying of arrays in GNU
APL. There were some suspicions<br>
claimed earlier that some copying could be avoided. But it
then turned out that removing these<br>
copies would corrupt other values under certain
circumstances (because different functions would<br>
modify the same value). We therefore had to revert the
"unnecessary copies" to a state that seems<br>
to be safe now.<br>
<br>
Regarding parallel processing, the situation seems to be so
that </font><font face="Helvetica, Arial, sans-serif"><font
face="Helvetica, Arial, sans-serif">a multi-core CPU
cannot be<br>
significantly faster than a single-core CPU with the same
memory (-interface). A significant<br>
speedup requires that the bandwidth between the cores and
the memory grows with the number<br>
of cores. I had built a machine like that with some
students back in 1990, but the current PCs with<br>
multi-core CPUs just do not provide that.<br>
<br>
If you <b>make parallel1</b> and then run <b>ScalarBenchmark.apl </b>
(which comes with GNU APL)<br>
then you get, for example:<br>
<br>
</font></font><font face="Helvetica, Arial, sans-serif"><font
face="Helvetica, Arial, sans-serif"><font face="Courier
New, Courier, monospace"><b>-------------- Mix_IRC +
Mix1_IRC -------------- </b><b><br>
</b><b>average sequential startup cost: 530 cycles</b><b><br>
</b><b>average parallel startup cost: 1300 cycles</b><b><br>
</b><b>per item cost sequential: 122 cycles</b><b><br>
</b><b>per item cost parallel: 195 cycles</b><b><br>
</b><b>parallel break-even length: not reached</b></font><br>
<br>
This means that:<br>
<br>
the additional start-up cost for a parallel computation
is 1300-530=770 cycles or 240 nano-seconds<br>
on a 3.2 GHz machine. This is actually a pretty good
value. Before writing my own core synchronization
functions I used a standard paralle;ization library that
took aboutc 20,000 cycles.<br>
I believe it was libmpi but that I was many years ago so I
don't quite remember.<br>
<br>
This startup cost also includes what Elias refers to as
bookkeeping).<br>
<br>
What remains (and is the show-stopper) is the per item
cost A single core needs 122 cycles for<br>
adding two numbers while 4 cores need 195 cycles per core
= 780 cycles in total.<br>
The code in both cases is exactly the same. Putting it
differenly, if I run alone then some function<br>
takes 122 cycles and when my collegues work in parallel on
something that has nothing<br>
to do with my work then I need 780 cycles.<br>
<br>
Once the parallel startup has finished the cores work
independently and without any locks or<br>
the like between them. This cannot be explained at
software level but rather suggests that<br>
some common resource (memory ?!) is slowing each core down
when some other core(s) are<br>
working at the same time. The 122 cycles (~40 ns) in the
single core case is roughly the time<br>
for one DRAM access in page mode. In other words, the main
memory bandwidth (at least of<br>
my machine) is just enough to feed one core, but far to
low for 4 cores.<br>
<br>
GNU APL can do little to fix this bottleneck. I believe I
have done everything possible (although<br>
new ideas are welcome) that can be done in software, but
if we hit hardware bottlenecks then<br>
then thats it. I believe GNU APL would run perfectly on a
4-core CPU with a 4-port main memory,<br>
but that will probably remain a dream in my lifetime.<br>
<br>
Best Regards,<br>
Jürgen Sauermann<br>
<br>
<br>
</font></font><br>
<div class="moz-cite-prefix">On 07/04/2017 06:15 PM, Blake
McBride wrote:<br>
</div>
<blockquote
cite="mid:CABwHSOtn3S-4s8KUTTziYayOrPq6qzv--W4tZCnfwf=***@mail.gmail.com"
type="cite">
<div dir="ltr">Hi Ellas,
<div><br>
</div>
<div>I remember your earlier work and comments. In fact,
that is the source of my knowledge of this.</div>
<div><br>
</div>
<div>Putting more features on a base with known issues is
problematic for the following reasons:</div>
<div><br>
</div>
<div>1. The time and effort it would take to secure the
base is being spent elsewhere.</div>
<div><br>
</div>
<div>2. Adding new features is, in effect, ignoring the
problem and moving on.</div>
<div><br>
</div>
<div>3. APL is good and useful as it is. If it's not
being used as-is, then there is some other issue (like
efficiency). I remember another post about unnecessary
bloat because of the unnecessary copying. This was
causing his machine to page unnecessarily. The
unnecessary coping is major in terms of performance and
memory efficiency. This is big.</div>
<div><br>
</div>
<div>4. APL was developed with the promise of parallel
processing. This is something that wasn't practical
back then, but is now. I am under the impression that
this was one of the big reasons Juergen created GNU APL
- to take advantage of parallel processing.</div>
<div><br>
</div>
<div>Just one opinion.</div>
<div><br>
</div>
<div>Blake</div>
<div><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jul 4, 2017 at 11:01
AM, Elias Mårtenson <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:***@gmail.com" target="_blank">***@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="auto">Hello Blake,
<div dir="auto"><br>
</div>
<div dir="auto">I agree those are important points
too. Some time ago, I spent time trying to
improve those things. Unfortunately, my approach
turned out to be a dead end. I have some other
ideas but those require a significantly
different underlying architecture. Essentially,
it requires a garbage collector, which changes
things quite a bit. </div>
<div dir="auto"><br>
</div>
<div dir="auto">I'm sure Jürgen can go into more
detail, but my impression is that avoiding
copying is much harder than it seems. </div>
<div dir="auto"><br>
</div>
<div dir="auto">Regards, </div>
<span class="HOEnZb"><font color="#888888">
<div dir="auto">Elias </div>
</font></span></div>
<div class="HOEnZb">
<div class="h5">
<div class="gmail_extra"><br>
<div class="gmail_quote">On 4 Jul 2017 22:57,
"Blake McBride" <<a
moz-do-not-send="true"
href="mailto:***@gmail.com"
target="_blank">***@gmail.com</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div dir="ltr">My list would be different:
<div><br>
</div>
<div>1. don't clone arrays
unnecessarily</div>
<div><br>
</div>
<div>2. improve support for parallel
processing</div>
<div><br>
</div>
<div>With these, GNU APL would be much
more efficient. I think moving the
focus to CPU and memory efficiency is
much more important than adding
extensions of any sort.</div>
<div><br>
</div>
<div>--blake</div>
<div><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Jul
4, 2017 at 1:59 AM, Elias Mårtenson
<span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:***@gmail.com"
target="_blank">***@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="ltr">Thank you Jürgen,
<div><br>
</div>
<div>I think we understand each
others positions, and agree
that they are not entirely the
same.</div>
<div><br>
</div>
<div>That said, your points are
very well taken, and for the
most part I actually agree
with you.</div>
<div><br>
</div>
<div>I have a wishlist of
features that I personally
believe are important. For the
most part, these have been
discussed previously but here
they are for completeness
sake:</div>
<div>
<ol>
<li>Bignums</li>
<li>Lexical binding.</li>
<li>First-class functions.</li>
<li>Rational numbers</li>
<li>Some kind of easy-to-use
imperative structure (i.e.
something better than the
horrific :If :Then :Else
structure in Dyalog)</li>
<li>Some kind of complex
datastructure (again,
something better than the
Dyalog classes)</li>
</ol>
</div>
<div class="gmail_extra">Note
that out of these, the only
feature that would change the
semantics compared to the ISO
spec is the first, bignums. At
least if it's implemented in
the "natural" way.</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">I've
considered working on some of
these myself, but I have no
intention of doing so if
you're against these ideas in
principle. I certainly have no
desire to maintain my own
version.</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">Regards,</div>
<div class="gmail_extra">Elias</div>
<div>
<div
class="m_-5618828594290745300m_7199153563756770333h5">
<div class="gmail_extra"><br>
<div class="gmail_quote">On
4 July 2017 at 03:21,
Juergen Sauermann <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:***@t-online.de" target="_blank">***@t-online.de</a><wbr>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000"> <font
face="Helvetica,
Arial, sans-serif">Hi
Elias,<br>
<br>
thanks for
explaining your
position.<br>
<br>
My concern about
free software is
not so much
political but more
practical.<br>
<br>
If I look at
programming
languages, then my
impression is that
those languages
that make the<br>
exchange of
software simple
are more
successful than
those that do not.<br>
<br>
Historically it
has always been
possible to
exchange APL
software from one
interpreter to
another,<br>
but it was never
easy. Most of the
code can be
exchanged via <b>.ATF</b>
files, but the
problems were<br>
often tiny
incompatibilities.
These
incompatibilities
are spread all
over the code, so
getting some<br>
APL workspace to
work on a
different machine
is still an
adventure.<br>
<br>
That is why I
prefer to stick to
the ISO standard,
no matter how bad
it is. As long as
you use only<br>
standardised APL
functions you have
very few
compatibility
problems. There
are some, but they<br>
are well known.
But every new
function that is
not standardised
moves you away
from portability.<br>
If I object to
implementing some
new function than
this is not for
political reasons,
but because I<br>
am afraid that the
additional
incompatibility
makes the exchange
of APL software
more difficult.<br>
<br>
In some cases the
function is so
important that the
incompatibility
has to be
accepted. Examples<br>
for that are
certainly ⎕SQL,
⎕FIO, and maybe
dyadic ⎕CR and
⎕DLX. These
functions are
almost<br>
impossible to
implement
efficiently by
APL's own means.<br>
<br>
On the other end
(from my point of
view) we have
things like the
KEY function. I
still believe that
it<br>
rather fits into
the FinAPL Idiom
Library than into
GNU APL. It is
shorter than one
APL line and if<br>
you make it an
idiom then it
remains portable
between all APL
interpreters while
otherwise it is
only<br>
portable between
GNU APL and Dyalog
APL.<br>
<br>
I am open to
implementing a
feature if it is
really useful, but
only then.
Becoming a leader
in <br>
implementing new
feature is not one
of my priorities.
There are enough
other APLs that
are<br>
keen on that (e.g.
Dyalog and NARS,
see <a
moz-do-not-send="true"
class="m_-5618828594290745300m_7199153563756770333m_4572084747630161882m_1676935410655698271moz-txt-link-freetext"
href="http://www.nars2000.org" target="_blank">http://www.nars2000.org</a>).
The ambition of
GNU APL<br>
has always been to
become a stable
standard
interpreter some
day. That is
difficult enough,
and<br>
we have learned
from PL/I how too
many features can
kill a language.
And I have seen
too many<br>
software projects
that failed due to
being overly
ambitious. I
simply do not want
to share their
fate.<br>
<br>
Regarding emacs, I
can't help to note
that I am not
using it, because
it is, for my
taste, too
complex.<br>
I rather prefer
something simpler
like vi. Sometimes
less is just more.<br>
<br>
Best Regards,<br>
/// Jürgen<br>
<br>
</font>
<div>
<div
class="m_-5618828594290745300m_7199153563756770333m_4572084747630161882h5"><br>
<div
class="m_-5618828594290745300m_7199153563756770333m_4572084747630161882m_1676935410655698271moz-cite-prefix">On
07/03/2017
04:00 PM,
Elias
Mårtenson
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">Hello
Jürgen, and
thanks for
your thorough
reply.
<div><br>
</div>
<div>In terms
of the
usefulness of
Key, I don't
disagree with
you. I'd
certainly like
to see even
more flexible
solutions.</div>
<div><br>
</div>
<div>Where we
do disagree is
what the goal
of free
software is.
Arguably there
are probably
as many goals
as there are
people.</div>
<div><br>
</div>
<div>What
follows below
is an
explanation as
to why I
disagree with
your
assessment as
to what is the
best for Free
Software.
Please don't
take it as
personal
criticism. You
know that I
have the
deepest
respect for
you as the
maintainer and
author of GNU
APL.</div>
<div><br>
</div>
<div>After
spending quite
some time on
the Emacs
Development
mailing list,
I have learned
quite a bit
about what the
FSF's goals
are with
regards to
what they call
"Free
Software".
Time and time
again, RMS has
stated that
the goal of
GNU is to make
people use
commercial
software less.
In order
words, if a
project can
implement a
feature that
draws people
away from
commercial
software
towards Free
Software, then
that is what
the project
should do.</div>
<div><br>
</div>
<div>At this
point, I'd
like to
clarify that I
am not
completely in
agreement with
RMS on this.
In the Emacs
project, this
position has
prevented
Emacs from
gaining
certain
important
features,
simply because
they would
have made it
easier to use
"non-free"
software
together with
Emacs. This is
a position I
don't agree
with.</div>
<div><br>
</div>
<div>I'd
really like to
see GNU APL
become a
leader in
implementing
new features.
That way
perhaps we get
more people to
switch. The
point I'm
making here is
that by
implementing
useful
features that
would make
people choose
GNU APL before
any
alternative,
then the
project would
better serve
the GNU
goals. </div>
<div><br>
</div>
<div>Regards,</div>
<div>Elias</div>
<div><br>
</div>
<div
class="gmail_extra">
<div
class="gmail_quote">On
3 July 2017 at
21:36, Juergen
Sauermann <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:***@t-online.de"
target="_blank">***@t-online.de</a><wbr>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
Hi Elias,<br>
<br>
thaks. The
explanation is
a bit clearer
but the
problems
remain.<br>
<br>
Key is a
non-standard
APL function
and we should
be careful
with the
implementation<br>
of
non-standard
functions.<br>
<br>
Every function
in GNU APL is
an invitation
to use it. If
the function
is obviously
useful then it
improves<br>
the language.
If it merely
solves a
particular
programming
case, then it
may improve
GNU APL a
little,<br>
but at the
price of
incompatibility.
Programs using
it become less
portable and
that
undermines the<br>
goal of free
software.<br>
<br>
So the
question in
such cases is
how useful is
a function and
is that
usefulness
worth the
incompatibility?<br>
<br>
In the case of
the key
function I
would say no.<br>
<br>
First of all
the key
function can
only be used
if the data it
operates on is
organized in a
specific way:
that<br>
the first
column is the
key. That may
be the case
but the fact
that this is
needed is
somewhat
contrary to<br>
how other APL
function work.
You could also
call that
arbitrary.<br>
<br>
That goal can
easily
achieved by
other means.
If I have a
single <b>KEY</b>
then something
along the
lines of<br>
<br>
<font
face="Courier
New, Courier,
monospace"><b>((DATA[1;]≡KEY)⌿KEY)[1;]</b></font><br>
<br>
will give me
the first row
(or all rows
if I remove
the right
[1;]) in an
array that has
that KEY. I
suppose that
is<br>
more or less
what the key
function does
(plus applying
some function
on that
expression).
The expression
is<br>
even superior
to a function
because it can
be used at the
left side of
an assignment.<br>
<br>
If that is so
then the key
function is
only one of
several APL
idioms (see <a
moz-do-not-send="true" href="http://aplwiki.com/FinnAplIdiomLibrary"
target="_blank">http://aplwiki.com/FinnAplIdio<wbr>mLibrary</a><br>
for a rather
famous list of
more than 700
such idioms).
Each of the
700+ idioms
is useful and
would deserver<br>
its own
symbol, but if
we would do so
(which is
technically
possible due
to Unicode)
then we would
have turned<br>
GNU APL into
an unreadable
mess.<br>
<br>
Best Regards,<br>
Jürgen
Sauermann
<div>
<div
class="m_-5618828594290745300m_7199153563756770333m_4572084747630161882m_1676935410655698271gmail-h5"><br>
<br>
<br>
<br>
<div
class="m_-5618828594290745300m_7199153563756770333m_4572084747630161882m_1676935410655698271gmail-m_-4651243352051710126moz-cite-prefix">On
07/03/2017
05:50 AM,
Elias
Mårtenson
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">The
key function
is better
described in
the Dyalog
reference
manual, on
page 153
here: <a
moz-do-not-send="true"
href="http://docs.dyalog.com/16.0/Dyalog%20APL%20Language%20Reference%20Guide.pdf"
target="_blank">http://docs.dyalog.com/1<wbr>6.0/Dyalog%20APL%20Language%20<wbr>Reference%20Guide.pdf</a><br>
<div><br>
</div>
<div>Essentially,
it's a
grouping
function. It's
used to create
groups of
similar
things, and
apply a
function on
the individual
instances. The
examples in
the section I
referenced
above should
be pretty
clear, I
think.</div>
<div><br>
</div>
<div>Regards,</div>
<div>Elias</div>
</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
3 July 2017 at
00:51, Juergen
Sauermann <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:***@t-online.de"
target="_blank">***@t-online.de</a><wbr>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
<font
face="Helvetica,
Arial,
sans-serif">Hi
Elias,<br>
<br>
I am not quite
in favour of
it and it has
problems.<br>
<br>
It is not on
my keyboard
(even though I
am using a
Dyalog
keyboard).<br>
Not to talk
about other
keyboards.<br>
<br>
It does not
really look
like
need-to-have
function and I
suppose it can
be<br>
efficiently
performed by a
short
combination of
other APL
primitives.<br>
<br>
In my opinion
adding
primitives for
every
imaginable use
case (and<br>
there are
certainly use
cases for the
key function)
leads to an
overloading<br>
of the APL
language in
the long run
and does not
improve the
language.<br>
<br>
Another
problem is
that after
reading the
description
several times,
I still<br>
can't explain
in simple
terms what the
function is
actually
doing. That
makes it<br>
a good
candidate for
a never used
function if it
should ever be
implemented.<br>
<br>
Best Regards,<br>
Jürgen
Sauermann<br>
<br>
<br>
<br>
</font>
<div>
<div
class="m_-5618828594290745300m_7199153563756770333m_4572084747630161882m_1676935410655698271gmail-m_-4651243352051710126h5"><br>
<div
class="m_-5618828594290745300m_7199153563756770333m_4572084747630161882m_1676935410655698271gmail-m_-4651243352051710126m_-9156086468235745953moz-cite-prefix">On
07/02/2017
06:24 PM,
Elias
Mårtenson
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">How
about
implementing
the key
function, ⌸?
<div><br>
</div>
<div>It's
described in
this article
on the Dyalog
site: <a
moz-do-not-send="true"
href="https://www.dyalog.com/blog/2015/04/exploring-key/"
target="_blank">https://www.dyalog.com/blog/20<wbr>15/04/exploring-key/</a></div>
<div><br>
</div>
<div><span
style="font-size:12.8px">Jürgen,
are you in
favour of this
function?</span><br>
</div>
<div><br>
</div>
<div>Regards,</div>
<div>Elias</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</blockquote>
<br>
</body>
</html>