common lisp computer science recommended literature

recommended reading: https://irclog.tymoon.eu/search?channel=libera%2F%23commonlisp #commonlisp chatlog (search for keywords such as OOP, database, protocol, software architecture, etc)
CLIM II specification ("beach I learned object-oriented programming in Common Lisp from the CLIM II specification."), the writings of Bertrand Meyer

beach jmercouris: You should definitely avoid languages that silently hide problems in application code.
beach jmercouris: There is lots of incompetence in software development.
phoe I prefer my code to fail fast, and type errors in dynamic programming are exactly where programs should fail fast
 beach jmercouris: phoe puts it well.  Incorrect code should fail sooner rather than later.
beach jmercouris: As an example of making the language help you, it used to be the case that (setq <var> <form>) at the REPL would silently create variable <var> if it did not already exist.  That is a terrible idea, because a spelling mistake in a variable will then go unnoticed.
beach jmercouris: I suggest you read the writings of Bertrand Meyer.
He formalized something known as "programming by contract" which is basically the idea of catching errors as soon as possible.  Before him, there was a tendency to hide errors.
reading his stuff at the time was an eyeopener for me.  Before that, a program that crashed was considered the worst thing that could happen.  After his writing, we understood that it was better for a program to crash (soon) rather than silently giving the wrong answer.
Start with the Wikipedia article on "Design by contract"
https://en.wikipedia.org/wiki/Design_by_contract
 beach I know it can be hard to motivate oneself to study these things, especially when one is "lost in a sea of mediocrity" as Dream Theater puts it.
jmercouris that's actually why I started with CL, I wanted to learn more and more thoroughly understand programming
beach I totally agree.  That's why I hang out here too.  There are some very smart and some very knowledgeable people here.
phoe the collective wisdom of the people who talk here is massive and wonderful



beach elderK: You don't need anything like forward declarations if you use generic functions.  The DEFGENERIC form plays the role of a declaration.
 A CLOS protocol (generalization of interface) consists of generic functions, classes, and initargs.
You might have a look at the CLIM II specification for how that works.  I learned object-oriented programming in Common Lisp from the CLIM II specification.






phoe jmercouris: if you write DEFMETHOD INITIALIZE-INSTANCE {:AROUND,:BEFORE,:AFTER}, then it MEANS that this is a Lisp constructor.
THIS is literally the idiom, just like Class(...) { ... } in java
if MAKE-INSTANCE is the analogue of java's new ..., then INITIALIZE-INSTANCE is the way to declare constructors
in :BEFORE I usually place keyword argument type validation, in :AFTER some logical validation that needs to have the slots already bound and then some additional stuff that needs to be computed
this way, when the slots become bound, they are always of proper type/contain proper values/etc.. and the stuff in :AFTER can assume that it operates on sane data



xificurC mfiano the book Let Over Lambda went really hardcore into some custom closure-based OO design and I didn't really understand why would one want to do that instead of using CLOS. The macros were interesting and the sorting networks too

scymtym shka_: inspect a complex number :)

beach xificurC: One absolutely wouldn't want to do that.

beach ... closure-based OO design that is.




White_Flame having to pollute all your objects with logging functions, for instance, is a massive ugly nuisance, as opposed to letting your logger peek in as deeply as it needs to




jmes Is it possible to have a generic function with some parameters that are only needed for one of its methods? e.g. I might want (draw object-with-coords) and (draw object-without-coords x y). Does this make any sense?

jmes Let's say I don't want to make a subclass of object-without-coords to hold its coords

beach clhs 7.6.4

specbot Congruent Lambda-lists for all Methods of a Generic Function: http://www.lispworks.com/reference/HyperSpec/Body/07_fd.htm

beach You can use &REST for that.

beach (defgeneric draw (object &rest coordinates))

beach ACTION now sees a change of requirements coming.

_death how about &key &allow-other-keys

beach Then you can't call it as the second example shows.

_death true

jmes beach: thanks I think that is what I needed.

beach Either way, it is bad protocol design.

jmes Yeah, I would rather have the subclass but I'm just juggling some design constraints

jmes and my lack of CL knowledge

beach I would have two different generic functions.




Guest7491 is there any consistency in drawing apis arguments or expected args. (rectangle x1 y1 x2 y2...)(rectangle x1 y1 width height...)(rectangle pt1 pt2...)...
beach You may check what CLIM does.  It often has two functions like DRAW-RECTANGLE and DRAW-RECTANGLE*.
beach ... depending on the arguments are points or single coordinates.
there are plenty of other examples in Lisp (even before Common Lisp) where * means "spread arguments".
The * versions use single coordinates, so they are the "spread" versions.
Section 2.3 in the CLIM specification.
For what it's worth, every "regular lisp programmer" should read the CLIM II specification.  It's a good tutorial on the use of CLOS-style object-oriented programming.  Plus, McCLIM is a very good implementation of it that I encourage GUI programmers to use.
 I myself understood the virtues of CLOS-style object-oriented programming from reading the CLIM II.  Reading Keene's book was just not enough.  And since I rarely think of myself as unique, I think that any Common Lisp programmer who wants to use CLOS seriously, should read the CLIM II specification.
I found that the concept of a protocol was made clear by the CLIM spec.  For example that the important parts of a class are the initialization arguments and the functions that can be applied to instances of it.  So the slots are implementation details.
 Also the use of auxiliary methods in the drawing functions.

beach McParen: I don't know the answer to your question, but it is a bit strange to want to document a slot.
beach McParen: Slots are implementation details, so the only person interested in the role of a slot would be the person reading the code, and then a comment is the appropriate way to describe what the slot is for.


beach tempate: Common Lisp classes don't have methods in them.  Encapsulation is accomplished with an orthogonal mechanism known as "packages".
White_Flame (and packages aren't really "encapsulation" because you can always still poke into them; there's no real data hiding in CL, besides closures (and you can use implementation tools to poke into those, too))

beach Unix code, for instance, silently gave (gives?) the wrong answers.  One of the rules that RMS established for the GNU project was to avoid that.
beach jmercouris: As an example of making the language 'help' you, it used to be the case that (setq <var> <form>) at the REPL would silently create variable <var> if it did not already exist.  That is a terrible idea, because a spelling mistake in a variable will then go unnoticed.
sjl_ re: setq autocreating variables and making typos unnoticed, this why I find defmethod not requiring a defgeneric to be annoying
beach I totally agree.
I have spent many hours debugging spelling mistakes in DEFMETHOD.
 sjl_ Or forgetting a package prefix on the method name.  That one is extra fun.
beach Yes, that one too.
Interestingly, in the book "Object-Oriented Programming, The CLOS Perspective", one of the authors states that DEFMETHOD was meant to be the primary interface.   I think that phrase shows the age of the book.
phoe (defmethod frobnicate ((object foo)) ...) (defmethod fronbicate ((object bar)) ...) ; look ma, two different GFs
dlowe (defun foo-frobnicate (foo) ...) (defun bar-fornbicate (bar) ...) seems equally likely and bad.
beach dlowe: (bar-frobnicate ...) will result in an error.
splittist dlowe: if you call BAR-FROBNICATE you get an undefined. If you call FROBNICATE with a BAR, anything could happen, depending on your object hierarchy.
 dlowe ah, I see what you mean
phoe dlowe: the compiler will warn you about undefined functions. it won't warn you about undefined GFs
beach But I think it should. And apparently, SBCL used to warn.
splittist how about a whole new compiler? How hard could it be?
beach Piece of cake.  I am on it...
splittist And a program editor, of course.
splittist With an OS to fit them together. Next week is fine.
beach Oh, man, I have articles to update.  Can it wait another week?
splittist I'm feeling generous - take 10 days.
beach Whew!  Thanks!

4:32:13
beach dbotton: The book "Object-oriented programming.  The CLOS perspective." mentions that DEFMETHOD was intended to be the main interface to the object system.  But in hindsight that was a mistake.
4:32:18
beach It is a very common mistake to misspell the name of a generic function in a DEFMETHOD form, or to omit a required package prefix.  Then a new generic function is silently created, and the programmer has a bug that can be difficult to track down.
4:32:22
beach For that reason, I am in favor of the implementation emitting a warning (STYLE-WARNING I guess) whenever a DEFMETHOD form is encountered that mentions the name of a generic function that has not previously been defined.
4:32:23
beach As I understand it, SBCL used to emit such a warning, but it was removed according to request from the users.  I guess those users never make typos.

 beach If you allow for subclasses, then you have a CLOS protocol, and you should document what the protocol does and what it expects.  And that involves a set of generic functions together with documentation.  A method specialized to the base class is an implementation detail.
A CLOS protocol is a set of generic functions and possibly a set of protocol classes (not meant to be instantiated).  Such a protocol is what you expose to the outside world, together with documentation about how these entities fit together.
So by announcing the existence of a method, you are essentially (again?) writing code as if the class is the external protocol to be documented, but in CLOS, the external interface is a bunch of generic functions.

I always create a DEFGENERIC form for every slot accessor I have.
http://metamodular.com/cluffer.pdf
Look at appendix A.
As you can see, there is a set of protocol classes and a set of generic functions.
Forget about the internal protocols for now.
(beach I am not that much in favor of documentation strings.
beach I prefer real documentation.
beach But that's a different discussion.)
Frequently, a protocol will have protocol classes for which it provides little or no functionality, and then it will have implementation classes that are subclasses of the protocol classes (often named standard-...).  The methods of the module will be specialized to the implementation classes rather than to the protocol classes (with some exceptions).
This technique allows client code to subclass the protocol classes to avoid the default implementation provided by the module.
Technically, you should also document the method combination and the argument precedence order of your generic functions. And those are things you can't express with DEFMETHOD. Though, arguably, by default it is the STANDARD method combination, and the argument precedence order is the default. The method combination becomes part of the protocol to be documented (if you use something else than the STANDARD one).
In traditional object-oriented languages, you only have one method combination, which is a subset of what the Common Lisp STANDARD method combination does.
My point is that the DEFGENERIC form is the place where you can attach things like the method combination being used, the argument precedence order, the method class, the documentation, etc.
So the generic function is the thing that should be exposed to the client as part of the protocol.


beach "object" means "any Lisp datum" as the Common Lisp HyperSpec explains.
"standard object" is what you typically get when you instantiate a class defined using DEFCLASS.
So 234 is an object.
But not a standard object.
clhs standard-object
specbot http://www.lispworks.com/reference/HyperSpec/Body/t_std_ob.htm

flip214 beach: can autogenerated accessor functions be documented via DEFCLASS?
phoe flip214: I'd DEFGENERIC :DOCUMENTATION them myself
phoe (which would be better for the protocol, anyway)
phoe then DEFCLASS will just slap methods on top of these functions.
beach flip214: I don't know, but you should always do the DEFGENERIC in my opinion.
beach What phoe said.
beach Just having a lambda list that you determined yourself, as opposed to OBJECT is worth it.



beach elderK: You don't need anything like forward declarations if you use generic functions.  The DEFGENERIC form plays the role of a declaration.
 A CLOS protocol (generalization of interface) consists of generic functions, classes, and initargs.
You might have a look at the CLIM II specification for how that works.  I learned object-oriented programming in Common Lisp from the CLIM II specification.



beach elderK: I put my DEFGENERIC forms in a separate file.

beach More important is what package you use.

beach elderK: I create "modules" consisting of an ASDF system definition, a package definition, and a bunch of files with code in them.

beach If possible, one such file contains the DEFGENERIC forms.

elderK beach: How would you deal with the situation where say, the continuation module depends on the expression module, which depends on the continuation module?

beach You would not design modules that way.

elderK Agreed.

elderK This is, however, the way the code in the LISP book is structured, unfortunately. Although, it doesn't use CL nor namespaces.

beach Then they would probably be in the same module.

beach What book is that?

elderK Lisp in Small Pieces.




schweers Is Sonya Keenes book on CLOS really that good?  Especially if one has already read and understood the relevant chapters from PCL and uses CLOS casually?

beach Probably no great point in reading it then.

schweers okay, thanks

schweers I somehow struggle with proper OO design given generic functions and MI.  I always thought that “OO design” was a fad, and think that it is in mainstream languages.  But thanks to CLOS I can see what I’m missing.

schweers Do you (or anyone else) have a good book recommendation?

beach I learned object-oriented design in Common Lisp by reading the CLIM II specification.

schweers oh

Shinmera The book of: read code that uses CLOS a lot (like beach's stuff)

beach Before that I had read Sonja Keene's book, but I guess I didn't "get" it.

beach Yes, that too.  You can look at Cluffer for instance.  It has documentation and tests too.
Shinmera Some of my stuff also uses CLOS heavily (and some of it the MOP heavily)
phoe I agree - Shinmera's code uses a lot of OO code and OO patterns in general
schweers thanks for the tip, especially about cluffer.  I see that you have 35 pdf pages worth of docs :)


phoe I actually learned a lot about object composition by reading and analyzing Shinmera's parachute library
beach schweers: I do recommend the CLIM II specification.  It is not quite as good as the Common Lisp HyperSpec, but the low-level parts are quite well specified.
phoe take it slow with the CLIM documentation - just start in one point and try to understand what it does, and then expand it by reading about an adjacent class/function/thing

schweers I think I like the idea of changing the class in order to change which methods apply without changing object identity.
beach schweers: Thanks.  I have been contemplating this design for 30 years or so. :)
schweers wow


beach In good, modular, object-oriented design, you always start by designing a protocol, i.e. a set of generic functions.  Only later do you decide what functionality should be provided in the form of slots, and what functionality should be computed.
phoe_krk Hmm.
beach Slots are an implementation detail that you don't want to decide right away.
beach That way, you can change the decision later, depending on factors such as performance.
phoe_krk I actually started with slots that later ended up being modified by generics anyway.
phoe_krk And now it's generics that are the most important when I wrote them as the second.
beach You should never start with the slots.  A protocol is a generalization of what is called an "interface" in less general languages, and that is what is important.
beach Furthermore, I can very well imagine implementations on which SLOT-VALUE and (SETF SLOT-VALUE) would be slower than an accessor.
phoe_krk DEFCLASS* suggests SLOT-NAME-OF accessors.
phoe_krk So, HASH slot, HASH-OF accessor.
beach That is one possibility.
beach But then again, it is not a good idea to generate the accessors automatically.
beach The other possibility is to do what I do, prefix the slot name with %.
beach The % character has a traditional meaning of "internal" or "danger, don't use".
beach Which is exactly what one wants to communicate to someone using slot names directly.
beach (defclass password () ((%hash ... :accessor hash)...))  something like that.
phoe_krk I see.
beach Someone trying to use slot-value would then have to type package%hash.
phoe_krk Which is scary enough.
beach and  and % both indicate "don't do it".
beach So when you use SLOT-VALUE you basically commit to having a slot and you make it impossible to have any other action when you access the slot.  With accessors, you can later add auxiliary methods, or change the implementation completely to using a calculation rather than a slot.

beach Why do your methods return T?
phoe_krk Inside password stuff, I don't want to return anything - not a hash, not a digest, not a salt.
beach Then use (VALUES) instead.
phoe_krk Oh! Thanks.
beach Returning T tells the maintainer of your code that someone is relying on that value when the method is called.
phoe_krk I see.
Grue` I think returning nil is also reasonable
beach It is definitely better than T, yes.

 phoe_krk How good would it be to basically make the password class write-only, with the only exception being a predicate PASSWORD-MATCHES-P?
beach That's a good solution.
phoe_krk That takes a password instance and a string?
phoe_krk Hum. And it simplifies things a real lot, too.

 phoe_krk Should I make a custom constructing function, say, MAKE-PASSWORD, or instead rely on bare MAKE-INSTANCE?
phoe_krk Because the second appeals more to me.
beach There is no consensus in that respect.
reepca isn't there a way to make make-instance do what you define?
beach Sonja Keene recommends using custom constructor functions.
reepca Why not make both work?
phoe_krk reepca: there is!
phoe_krk INITIALIZE-INSTANCE :AFTER.
phoe_krk And I think I'll actually go with a custom MAKE-PASSWORD function which will actually wrap around MAKE-INSTANCE that will then call INITIALIZE-INSTANCE :AFTER.
phoe_krk So both will work, like reepca said.
beach phoe_krk: You can't call INITIALIZE-INSTANCE yourself.
phoe_krk beach: I know! MAKE-INSTANCE will call it on its own.
beach Right.  Just wanted to make sure you know.




tempate What's wrong with slot-value?

beach It is very low level.  Usually, you design a data structure around classes and generic functions.  That's known as a "protocol".

beach A protocol is a generalization of an interface in other languages.

beach Just as you don't want to access fields in a structure or class directly, you typically don't want to access Common Lisp slots directly.

beach That's just basic software engineering.




beach Plazma: Consider the (artificial) example of displaying figures on windows.  In a message-passing system, you have to decide whether to make the display method part of the figure class or the window class.
beach Plazma: And that decision has to be made early, which is typically too early.
beach And you have to decide whether to write window.display(object) or object.display(window).
beach In Common Lisp, you just say (display object window), or whatever order you prefer.  It makes no difference.
beach And methods of the generic function can specialize on both parameters, as in (defmethod display ((object triangle) (window x11-window)) ...)

Plazma hm

Plazma interesting

beach Interesting, sure, but also invaluable and unique.

Plazma yea i'm trying to process it to be honest

beach I am betting that, when you learn more about CLOS, that your appreciation for message passing will vanish.

beach And multiple dispatch is but one feature of CLOS.  Auxiliary methods make some of the traditional design patterns disappear as well.

beach Method combinations come in very handy too.

Plazma probably, if i can keep up and actually start using it more, and try to balance being stuck in academic/python land

beach None of that stuff exists in traditional object-oriented languages.

beach Why would academia make you stuck with Python.  I thought academia was one place where you are free to explore and choose your tools.

beach No?

Plazma well that's what a lot of the scientific libraries are in

Plazma for machine learning, quantum computing, etc

beach I am sorry to hear that.
Plazma i mean you don't HAVE to, but yea.. and it's a mess
beach Are you sure about those libraries?  I mean, I would think those domains would need performance, and there is no way Python can provide that.  So I am guessing they are really written in C.  And if so, it would be even better to use them from Common Lisp, via CFFI or something like that.

Plazma beach: so it's more the API's themelves than anything, the libraries they reference are often originally in fortran or C, but there is something called CPython that it essenitally uses for the performance bits

Plazma i mean it's still slow as balls since a lot fo QC/ML is matrix math and mainpulations

Plazma and factoring

beach I am frankly amazed that academics would choose such a strategy.

Plazma they're lazy

Plazma i wish i was kidding honestly

beach Maybe scientists, like chemists, and physicists, but those domains are computing domains.

beach Oh, well.  I am glad I can choose my tools.

Plazma maths and physicists think they're good programmers and it shows :D

Plazma chemists too

Plazma but hey it works for everyone, somehow so

beach Sort of.  But it's a huge waste of money, effort, and brain power.

beach Anyway, we are drifting off topic.  So to summarize, CLOS is WAY better than traditional object-oriented languages.
beach Plazma: The thing with Common Lisp is that it does not have this silly idea that traditional object-oriented languages have, namely that classes are both for data representation and for encapsulation.  In Common Lisp, those aspects are handled by two different features.  Classes are for data representation, and packages are for encapsulation.
beach So in Common Lisp, saying that something is an "object" does not restrict what it is very much.  An integer is an object just like a stream or a symbol.  And you can write methods that specialize to integers, like (defmethod add3 ((x integer)) (+ x 3))

beach This is the sane way of doing object-oriented programming.
 beach No doubt aeth will disagree, but again, aeth is entitled to disagree.
Alfr_ beach, the default left-to-right one, I think you then could simply dispatches on the first match, in your example ((x some-type) y). And for (x (y class)), wouldn't (x t) simply only have to dispatch further on y's type?
beach You may be right.  I just have this hunch that cascading single dispatch can't do everything that multiple dispatch can.  Maybe some day I'll dig into it and write a paper about what I find.

contrapunctus Can client code add slots to an existing CLOS class, short of redefining the class entirely?
hayley Stealth mixins?
beach Adding a slot to a class basically amounts to redefining it.
beach contrapunctus: You would have to execute something like REINITIALIZE-INSTANCE on the class metaobject.
beach http://metamodular.com/CLOS-MOP/initialization-of-class-metaobjects2.html

contrapunctus hayley, beach: would it be terrible if, instead of that, I had a plist/alist as a slot? (I'm trying to implement arbitrary properties for objects.)
beach contrapunctus: I think it all depends on your requirements in terms of flexibility, power, etc.






moore33 beach: I admire your discipline in cutting yourself off from IRC during the day.
beach moore33: Thanks.  Yes, otherwise I wouldn't get anything done.



beach met`: What I call "uniform reference semantics" is the only sane way of programming.  Languages that distinguish between pointers and non-pointers quickly become way too complicated for application programming.


beach Common Lisp was designed so that a compiler can generate fast code, if written the right way.
As opposed to languages like Python that do not even pretend that it is possible to generate fast code.

beach matt__: In general (= x 0) or (= 0 x) should be expressed as (zerop x).  There is a very general rule in programming that says you should use the most specific construct that will do the job.  This rule is meant to make it faster for the reader to understand what is going on.

beach asarch: Unless you want to spend your software career doing silly stuff like designing web pages, you need to know some theory.
asarch When I was learning PostgreSQL and the relationships, a fellow from #NetBSD told I should learn Prolog or Lisp in order to "get the most" of the SQL design
 no-defun-allowed It may also be a stretch to say that SQL had a design; <http://www.pipeline.com/~hbaker1/letters/CACM-RelationalDatabases.html> claims that it was just a codification of what databases did then.

nwoob I'm trying to become good at programming. So i picked up common lisp, what other things should I take in parallel to improve myself. Please guide me.  just give me some guidance.
nwoob I generally want to become good in computer science. Should I learn basics of language and move to algorithms ?
nwoob cool will do that beach
 beach nwoob: I recommend reading up on abstract data types and then implementing them in Common Lisp.
Abstract data types are totally essential in computer science, and they will make a huge difference on the performance of your code.
But be careful, books (even my famous authors) sometimes contain sub-optimal or even incorrect algorithms for such data types.
So when you have a design, don't hesitate to show it here.
nwoob definitely, I seek advice from you guys
beach nwoob: Now, learning Common Lisp is a big project in itself.  In particular, learning to use generic functions and standard classes is going to take you some time.
beach nwoob: Implementing abstract data types will exercise your knowledge in modularity as well.  It will teach you how to separate interface and implementation.

 beach I found John Allen's book immensely useful, and a great inspiration.

beach To become an expert in just about anything takes around 10000 hours of practice, according to many sources.
jmercouris 10,000 hours, that's a long time
beach That's a necessary, but not sufficient, condition.  If you do it wrong, you will still be a beginner after 10k hours, like most professional software developers.
jmercouris Well, I think I've been definitely programming for over 10,000 hours
beach "In software development, people don't have 10 years of experience.  They have 1 year of experience 10 times."

shka_ i like this one :D
shka_ it is so true it hurts
jmercouris beach: what would you say separates a beginner programmer from an advanced one?
beach The amount of practice, of course?

beach But if you mean two such programmers with the same amount of practice, then "intellectual curiosity".
jmercouris No, that's not what I'm asking
jmercouris I'm saying something like, what is the difference between an expert and a beginner

whoman when you know the difference, you might not be beginner
jmercouris what kinds of things would a 1 year experience 10 times developer have
jmercouris vs one that had been constantly progressing
beach jmercouris: The one making progress has intellectual curiosity.
 beach jmercouris: So he or she will read code, articles, books, etc.  And he or she will try new things, fail, fix it, try again, etc.
jmercouris that's what they'll do sure, but what CAN they do that the other CANNOT
beach jmercouris: They can write high-quality maintainable code.  And they can do it faster.

 jmercouris I've worked with a lot of developers, and they seem to really know their domain, so idk, maybe it is the kinds of people I work with, or perhaps I am one of them and don't know it
 jmercouris But what does it mean to be a good engineer then?
beach jmercouris: Congratulations, you are very lucky.  For example, I myself consider it an absolute necessity for a developer who is supposed to write efficient code, to know all about what the compiler is capable of, so he or she needs to know everything about compiler design.
jmercouris I think I will disagree with that sentiment
 jmercouris should you just know a little of everything? or be an absolute expert in a domain
beach Of course we do.
jmercouris I think it can be advantageous to know the compiler, but also DETRIMENTAL to know the compiler
jmercouris the whole purpose of the compiler is to serve as an abstraction between lower level bytecode and what we're writing
as soon as we break that abstraction, we start writing code outside of the spirit of whatever language it is
 beach jmercouris: But I know very few developers who even care about what the compiler does.  Of course, there are jobs where it doesn't matter if you write efficient code or not.

shka_ jmercouris: i would rather say that the purpose of the language is to act as mere interface to compiler :]

jmercouris No, the purpose of the language is an abstraction
jmercouris it is not an interface to the compiler
jmercouris It is a made up paradigm, and we should respect the authors intent
jmercouris otherwise we'll work against the language
shka_ perhaps we should?
shka_ sometimes
beach jmercouris: I think you misunderstand my point.  What I see over and over again are developers who don't know what the compiler is capable of, so they write "optimized" code that is not necessary, and can even be worse than the "unoptimized" version.
shka_ just to see what happens
jmercouris beach: yes, I had unerstood something different
beach jmercouris: And, because they have no idea what the compiler is capable of, they either just guess, or they go by rumors that are sometimes several decades old.

jmercouris Yes, I've seen that many times in the python community
jmercouris where bad habits carry over from python 2 to python 3 and the performance of things is vastly different
shka_ anyway, I, personally, CONSTANTLY switch domain which comes with a price of being in "I have no idea what i am doing" state most of the time
beach shka_: That's the essence of learning, feeling like a child, basically.
 beach jmercouris: And I am betting that there are still people who replace functions by macros in C for reasons of efficiency, not knowing that 1. The VAX is no longer a relevant platform, and 2. The compiler can inline functions these days.

beach jmercouris: Here is another one for you: I think that every professional software developer should know Common Lisp.  Not in order to use it in their job, which is unlikely anyway, but in order to know more techniques for structuring code, and in order to know what they are missing when they use whatever they have to.
jmercouris beach: It's like tasting the nectar of the gods
beach So, in summary, a professional software developer who doesn't know about compiler design, and who doesn't know Common Lisp, is not a great professional software developer.
beach Not to mention data structures, algorithms, complexity theory, etc.
 beach And I bet the same people who "need all the performance they can get" are typically ignorant about those things.
 flip214 you'll need to know about various data _structures_ - linked lists, double-linked lists, arrays, hash-tables, binary trees, tries, skip-lists, bloom filters, etc.
 beach Most professional software developers don't need to implement balanced trees.  But they need to be "good consumers", i.e. knowing the performance characteristics so that they can choose the right data structure.  But they typically don't, as flip214 pointed out.
jmercouris I think my favorite thing to do is to invent data structures
 shka_ beach: well, some WANT to
beach shka_: Sure.  I am one of them.  My Cluffer library is an example of a data structure that I have been working on for a few decades.






beach One major advantage of Common Lisp is that a serious project can choose the language for a long-term project, knowing that the standard will remain as it is, that the language won't evolve according to the whim of some benevolent dictator who is ignorant both when it comes to programming language design and compiler technology.
beach ... and that there are several implementations, free or commercial according to the needs of the project, that implement that standard.
fiddlerwoaroof Yeah, I really wish more "modern" languages were implemented as compilers to CL
fiddlerwoaroof ACL2, Shen, etc. show that this is possible and CL is a much nicer target than C or lower-level stuff, if you don't need performance
fiddlerwoaroof (that is, if you're not in a domain where you need real-time guarantees, etc.
Jachy That still wouldn't help much though when the younger languages update and make breaking or incompatible changes...
beach fiddlerwoaroof: There are very few modern "languages", if by "language" you mean an independently published specification of the syntax and semantics of valid phrases, and the consequences of submitting invalid phrases to the compiler.

beach There are, however, plenty of "programming systems" without any attempt at a detailed specification, and certainly no specification of what will remain unchanged in the future and what might change at arbitrary times in arbitrary ways.


alexsotodev Has anyone explored the programming language Factor? It's object system is centered around CLOS-inspired generic functions in place of traditional message passing. It's documentation and developer tools like the listener and browser bear similarities to CLIM. More info below:
alexsotodev https://factorcode.org/
alexsotodev https://docs.factorcode.org/content/article-ui.html
alexsotodev https://docs.factorcode.org/content/article-ui-tools.html
alexsotodev https://docs.factorcode.org/content/article-ui-presentations.html
alexsotodev https://docs.factorcode.org/content/article-objects.html
beach In my talks to industry, I strongly discourage the use of any programming language without an independent standard or specification.  And I am guessing Factor does not have such a thing.  Of course, for hobby activity, it matters much less.
alexsotodev beach: Thank you, that's a valuable perspective; I don't believe Factor is based on a specification or standard.
beach Right.  Most programming languages aren't.  And most "languages" aren't even real languages, in that a single implementation defines the semantics, so the semantics can change between releases.
beach In my talks, I suggest that a project that decides to use a language without an independent standard should also plan for rewriting their code when the language semantics change, or to have a team of compiler experts that can independently maintain the language implementation if it should be abandoned, or if it should change in some radical way in the future.

 beach When I give talks to industry, I strongly advice against languages that do not have an independent standard.  Common Lisp is one of the few languages that that do that.  If a language does not have an independent standard, it can change at the whim of the person or organization that controls it, and your investment may be destroyed.

beach With Common Lisp, you also have several different, often very good, implementations available to you.  You can then choose the implementation that corresponds to your needs, and if one implementation ceases to be maintained, you can choose a different one with very few problems.

beach Furthermore, while an independent frozen standard would be a disaster to traditional languages, this is not the case for Common Lisp, since the language itself is extensible through its macro mechanism.

rainthree "Experimental languages are constantly moving tools with ever changing syntax and semantics. It becomes a huge chore to keep your code base up to date, and sooner or later you will stop trying. I have been through that cycle so many times before. Not just in OCaml. There is also RSI/IDL, C++ compilers ever since 1985, even C compilers.
rainthree The one constant, believe it or not, has been my Common Lisp. I’m still running code today, as part of my system environment, that I wrote back in 1990 and have never touched since then. It just continues to work. I don’t have one other example in another language where I can state that." - David McClain
rainthree "When I initially embarked on SML and OCaml, back in the late 1990’s, I was initially enthralled by the power of strong typing with inference. I was able to solve a nonlinear optimization over 150+ DOF using OCaml, whereas we had been stuck and bombing out after only 5 DOF using the strictly imperative Fortran-like language we had been using. I was so impressed, that word got out, and Phil Wadler invited me to write up my findings for his ACM Letters
rainthree , which I did.
rainthree And BTW… the breakthrough had absolutely nothing to do with typing and type inference. The success arose because of clean versus unclean control-flow. So you could say that OCaml adhered to “Structured Programming” in a better manner, which corresponds entirely to a fad cycle 2 layers back in time.
rainthree But then after several years of pushing forward with OCaml, in writing math analysis compilers and image recognition systems, I began to find that, despite proper typing and clean compiles, system level issues would arise and cause my programs to fail. The holy grail of provably correct code came tumbling down in the face of practical reality.
rainthree That’s when I began migrating back over to my old standby Lisp system. I live inside of my Lisp all day long, for days on end. It is a whole ecosystem. There is not crisp boundary of edit / compile / debug. It is all incremental and extensional. I think that kind of environment, regardless of language, is the holy grail of computing. I even had that kind of experience back in the late 1970’s when we were writing large telescope control systems
rainthree  in Forth." - David McClain


jcowan https://ai.googleblog.com/2006/06/extra-extra-read-all-about-it-nearly.html
(low + high) / 2;
In Programming Pearls Bentley says that the analogous line "sets m to the average of l and u, truncated down to the nearest integer." On the face of it, this assertion might appear correct, but it fails for large values of the int variables low and high. Specifically, it fails if the sum of low and high is greater than the maximum positive int value (231 - 1).
beach elderK: Oh that problem does not exist in Common Lisp.
We have bignums.
elderK beach: I'd still like to learn from it all the same, just to avoid making the same mistakes when I work in other languages.
beach I solve the problem by not working in other languages.
elderK :) I intend to do the same :)
jcowan Essentially all binary search implementations were wrong for *decades*
beach jcowan: They still are
 jcowan Unsurprising
elderK I'd be interested in learning why :)
beach elderK: In the past, they were wrong because they would fail in situations such as when the object did not exist, or when there were multiple occurrences.
beach elderK: Not so much now.  Now they just fail because they  start by testing for equality, which makes them take 50% longer than they should.
And if you start by checking for that, you increase the execution time by 50% on the average.
trittweiler You shouldn't test for equality at all. That's by definition only true once. It's the uncommon case :) (at most) once
beach trittweiler: Exactly.
beach And that's where most algorithms are wrong.
elderK I've been really investing serious time over the past few weeks, into CL.
trittweiler The other thing is that binary search should not be a binary thing, it should return the index of the item if found, or the index where the item would have to go if one wants it to be inserted. Of course in CL, there are multiple return values, so it easy and convenient to do that.
pjb A lot of bugs or problems that occur in other languages, (and that let people get real actual PhD and to build whole cottage industries earning hundreds of millions each year) just cannot happen in lisp.  This is why we're poor and with so few research grants.
jcowan The existence of bignums in Lisp is irrelevant, because binary search happens within an array, and the array size limit is always a fixnum.
pjb jcowan: depends on whether (<= (1- (* 2 array-total-size-limit)) most-positive-fixnum) or not.
jcowan No, because bignum arithmetic (no overflow) plus array bounds checking will give you an error instead of the wrong answer.
pjb jcowan: nope, because of /2
 pjb The problem is that with modular arithmetic, of when no overflow is detected, they get the wrong index
 beach This one is wrong too: https://fr.wikipedia.org/wiki/Recherche_dichotomique https://en.wikipedia.org/wiki/Binary_search_algorithm
beach Suppose I wrote a book on bridge building that systematically came up with designs that used twice the amount of material as required.  What would engineers think of that book.
beach p0a: So this binary-search thing is a very good example of what I mean.  People choose the wrong algorithm and lose a factor 2 or more.  Then they micro-optimize instead, gaining some 10% or so, thereby making the program bigger, harder to debug, and harder to maintain.
 beach Be careful with any old book on data structures and algorithms.  I estimate around half of published books get a simple thing like binary search wrong.
beach "wrong" in that the published version takes twice the time it needs to.
beach And sometimes "wrong" in that it doesn't work with duplicate elements, etc.

Josh_2 Tried asking if I could do my Uni work in a language other than C++, it's an algorithms module. Didn't go well, B&D it is.
beach Sorry to hear that.
beach C++ is a terrible choice for an algorithms course.
beach It is interesting to me that mathematicians and scientists, who are "perfection oriented" in their domains, are often "performance oriented' (or have a "closed mindset, as Carol Dweck says) when it comes to their computing tools.

White_Flame big-O notions are pretty degenerate at a collection size of 1 though
beach afidegnum: O(1) and O(n) is notation used in complexity theory.  The first one means that accessing an element takes constant time not matter how many elements there are, and the second means that accessing an element takes time proportional to the number of elements.
beach afidegnum: Interestingly, then, is that accessing an element of a Common Lisp vector is O(1), but in a Python list, it is probably O(log n) or something like that, since in Python a list is not a consecutive sequence of elements.
beach afidegnum: I strongly recommend you read up on things like this, or you run the risk of writing very inefficient code indeed, no matter what programming language you work in.
no-defun-allowed https://docs.python.org/3/faq/design.html#how-are-lists-implemented-in-cpython suggests they are implemented as adjustable vectors, but then inserting an element would have terrible performance.
beach That would be even worse, then, yes.
beach So that situation is very typical.  An inexperienced programmer uses some built-in collection type of some language, thinking that, because it is built-in, it is also fast.  Then they get terrible performance.
no-defun-allowed I don't think a Python programmer can tell the difference between "O(log n) but slow" and "O(n) but slow" though.
beach I think that's exactly my point.

beach I find Common Lisp perfect for designing good data structures.  In many languages, it is much harder to do so because of all the boilerplate code that is often required.
beach And in languages without automatic memory management, designing good data structures can be very hard indeed.



beach oni-on-ion: The issue isn't money.  I have offered to pay someone part time to help me out.  The issue is that it is hard to find someone with the right qualifications, the right motivation, and enough time.
oni-on-ion qualifications, ie. well versed in CL ?
beach And algorithms and data structures, and compiler design, and editor representation, and software engineering, and English writing skills.
beach warweasle: Money is not the problem.  Manpower is.
p_l aindilis: Xach's l1sp.org been pretty useful for me
p_l beach: money can be turned into manpower, though
beach p_l: Not as easily as one might think.  Not for this kind of work.
LdBeth Hire Lispers?
warweasle I think there are enough lispers already...We just don't work together well.
p_l beach: I'd say fifty/fifty. The problem is that it's non-trivial amount of money, and some of the skills necessary are kinda low availability on the market due to general IT stupidity
beach LdBeth: Most of the work I need done requires someone who is an expert in CLOS, in compiler design, and in software engineering,   I am sure there are Lispers out there that correspond to that list of requirements, but they are very likely employed, have families, houses, friends, etc.



beach It is getting better though.  Several people are pitching in.
beach But, let me say this again, even when we have something decent, it won't have any significant impact on the number of people using Common Lisp.
aindilis beach is there anything that would?
beach aindilis: Only a deep transformation about what is taught at universities.  And that's not going to happen because the teachers an insufficiently trained as well.


beach Josh_2: I often give talks to people who have been working for many years with software development.  But they have no idea about the difference between static and dynamic languages, and they have no clue how a static memory allocator works, nor how automatic memory management works.  They know nothing about compiler design, so they have no clue about the relationship between the code they write and what ends up being executed by the processor.
So they think that a dynamic language has to be interpreted, and they think manual memory management has no cost and no pauses, whereas they think that automatic memory management is expensive and gives long pauses.
They sometimes know about the difference between static and dynamic *typing*, but I am talking about the separation (or no separation) between compile time and run time.
schweers Probably the fault of our education system.
Nope, not only.
They just don't do any studying on their own.
They read no books, no papers, nothing.
schweers Oh right, years of experience.  Yeah, then it’s their fault.
schweers I still find an idea that Steve Yegge put forth at some time quite interesting.  According to him, many programmers only know a single language (or few languages which are very similar to each other).  They think that learning a second language will be as hard as learing the first.  During this time they will be unproductive and hence they fear for their livelihood.
 beach schweers: I don't think most developers even think that much about what they know and what they should learn.
beach schweers: I tell the industry people I give talk to, to take 4 hours per week for self study.
beach If their manager does not allow that, then the manager does not have the best interest of the company and its stakeholders in mind.

fwoaroof[m] I just need to sit down and write a compiler for a subset of lisp
fwoaroof[m] It's the sort of thing that I understand best just working through the problems
beach In my opinion, programmers need to know about compiler design.  Not so much in order to create fast code, but in order to avoid guessing what the compiler will do, and in the process, making their code less maintainable.  But there are cases, like this one, where such knowledge is needed for creating fast code.
fwoaroof[m] Yeah, I've been slowly discovering in my professional career why the old lisp books cover the topics they do
fwoaroof[m] Things like continuations and language design are surprisingly relevant to day-to-day programming
beach Exactly.
seok38 fwoaroof[m] yeah, for books on other specific languages I wouldn't trust stuff that is printed before 2012

beach maplambda: If you learn Common Lisp, it is unlikely that you would want to program in JavaScript later.
maplambda yea but i like money
beach maplambda: If you use SBCL (or any other Common Lisp implementation) without SLIME or something similar, you are likely going to hate it.
beach maplambda: Oh, then you should avoid learning Common Lisp, because you will never again be happy in your work, using another language.
ajithmk lol
maplambda i hope it lives up to all the hype
beach maplambda: You will end up like drmeister who wrote a Common Lisp implementation in C++ to avoid programming in C++, or like alandipert, who wrote a Common Lisp implementation in JavaScript to avoid programming in JavaScript.
maplambda will knowing lisp help me with understanding assembly and C and other languages ?
beach Oh, definitely.  It will help you understand lots of stuff about programming and programming languages.
maplambda ok cool
beach maplambda: But, more importantly, you will learn about programming techniques that you then wish you had at your disposal when you use other languages, like macros, generic functions, reader macros, compiler available at run-time, etc., etc.
maplambda are algorithms easier to write in lisp
maplambda like
maplambda sort, binary search, graph transversal etc
beach Easier than what language?  Clearly easier than in any language that uses manual memory management, sure.

maplambda i guess i mean intuitive or elegant
maplambda i never liked the way imperative languages implemented data structures
maplambda liked linked lists
maplambda ugly
maplambda pointers
maplambda ill never be a linked list/pointer kind of girl
sm2n those things are unrelated, common lisp is written in an imperative style quite often and it doesn't involve pointers directly usually
beach maplambda: I am sorry to bring you the bad news then.  Common Lisp is not a functional programming language.  It is a multi-paradigm language.  It does not have explicit pointers like C, but data structures look pretty much the same otherwise.
beach maplambda: And one of the central data structures in Common Lisp is a linked list.
beach LISP used to mean "LISt Programming"
beach maplambda: But, like I said, automatic memory management changes everything.
maplambda hm
beach maplambda: It allows your language to use what I call "universal reference semantics" which is a much simpler way of thinking about the semantics of your programs.
maplambda ok 1 last question what about speed
maplambda compared to say a "compiled language"
sm2n cl is a compiled language
maplambda sm2n, why am i using an interpreter then
sm2n good question, what are you using?
beach maplambda: There is no such thing as a compiled or an interpreted language.  It is a characteristic of the implementation.  Most modern Common Lisp systems compile to native code.
maplambda sm2n, sbcl
sm2n sbcl generates native code
beach maplambda: You are not using an interpreter.

maplambda so it compiles everything i type lol?
maplambda in a repl
sm2n you are confusing a repl with an interpreter
sm2n they are mostly unrelated, people have managed to hack repls in places you would be surprised, like C

maplambda actually that makes sense why would you want to compile the entire program at once given locality of reference
maplambda never thought of that before
beach maplambda: See, you already learned something by studying Common Lisp.
maplambda yea im convinced
maplambda damn
beach Try (defun f (x) x) and then (disassemble #'f)





beach That's part of what I was referring to.  The hypothesis was that, in order to simplify their work (design, first-time development, maintenance), since their main tool (i.e., the programming language) makes their task so hard, they need to cut down on the features.
beach What was so "interesting" about this presentation was that the speaker took features of C, C++, Perl, Awk, JavaScript, etc. and for each one stated why it was problematic.  Then he pretended to design a new and improved scripting language that fixed the problem.  It so happened that the fix was exactly what Python does.
beach However, what he failed to mention was that Lisp had fixed these problems ages ago, and in a better way than Python does.  Too bad that his fixes are going to result in a scripting language rather than a general-purpose programming language like Common Lisp.
beach I mean, I think he failed to mention that, not as a deliberate omission, but by simple ignorance.  Hence my analysis about the sorry state of the fields of computing.
aeth Doesn't Python have odd scoping rules?
ggole- It has function scope.
aeth Scope was solved by Scheme in 1975.
aeth So only C has an excuse there
beach I thought scope was solved by Algol60.
ggole- Arguably earlier in Algol, which had entirely sane lexical scoping rules
beach Exactly.
aeth Well, I guess the concept was at least "popularized by" Scheme
beach aeth: I would not attribute popularizing scoping rules to Scheme.  Pretty much every language invented in the spirit of Algo60 got it right.  Like Simula, Pascal, etc.

beach aeth: And, when I programmed in C, I considered local scope to be an important tool for limiting the portion of the code where variables are in scope.  It is (or should be)  an important tool for every developer in every language.  But, again, ignorance strikes in the field of computing.
beach I vividly recall teaching this tool to students in 1988, and I had to justify it by telling them about compiler design, because they were convinced that, if a variable were introduced inside a loop, then it somehow needed to be "created" for each iteration, thereby slowing down the code.
beach And this necessity is part of the reason why I think every serious developer must know about compiler technology, computer architecture, and programming-language design.
beach If they don't know about those things, they are going to make incorrect guesses, and the code is going to reflect those guesses in so many negative ways.
no-defun-allowed My C++ lecturer told us that not having to zero initialise variables makes C++ programs faster than other languages. I suppose data flow analysis wasn't invented that long ago...
beach ... which only goes to show that ignorance is not limited to students and developers.
beach Kildall's PhD dissertation on the subject dates from 1972.