Ideal for Scientific computing. ( LLVM based; Optionally typed; Dynamic )

https://julialang.org/

https://docs.julialang.org/en/v1/index.html

Julia Metaprogramming:

https://docs.julialang.org/en/v1/manual/metaprogramming/inde...

Julia: "Building a Language and Compiler for Machine Learning" ( compiling to GPU; TPU )

https://julialang.org/blog/2018/12/ml-language-compiler

"Why Does Julia Work So Well?"

https://ucidatascienceinitiative.github.io/IntroToJulia/Html... ( "Core Idea: Multiple Dispatch + Type Stability => Speed + Readability" )

I'm sure I do more of this than necessary, but much of my code ends up being "generated functions".

@generated foo(x, y) ... end

When you write an `@generated` function, you write code that creates a Julia expression. Using "x" and "y" in the function give you the types of these arguments. Defining parametric structs lets you pass any information you want (eg, array sizes) to specialize code and algorithms.

A very useful library: https://github.com/MikeInnes/MacroTools.jl

Flux is easily the coolest ML framework I’ve used, and I’ve just discovered Lazy.jl.

I didn’t really understand or get the point of metaprogramming, but the more Julia I’ve been writing the more I’ve seen fantastic uses of it and the more inspired I get.

Plus the language itself is a real pleasure to write.

Of course you can intercept pretty much anything with __dunder__methods (attribute missing, method access, instantiation, etc), you have full instrospection of pretty much anything (functions, objects, modules, call stacks...) and you also have monkey patching, decorators, metaclasses, bytecode injection...

But while we are far away from Lisp based languages, import hooks give you access to the ast of any module, and allow to use your own parser, before the result is loaded, to decide what to return. Which means you can pretty much make your own syntax and import it like a regular module.

However, the reason those tools are not very well known is that the community regards magic as dangerous as it is powerful, and everybody agrees on using it sparingly.

Hence the only popular libs that use a lot of magic are ORM (lot of metaclasses, dunders, etc) and test libs (monkey patching, bytecode injection, ast parsing), but not a lot more. I don't know of any popular lib that actually uses the import hooks to create a DSL.

Red's heritage comes from Lisp, Forth, and Logo. It's homoiconic and metacircular (it is its own meta language). Where Rebol was strictly interpreted, Red can also be compiled, and has hygienic macros. But you don't really need them. They're nice for moving things to compile time, but Red puts a twist on Lisp's sexpr model that obviates the "need" for them in most cases. Technically, you can say Red uses an fexpr model, but a more human-friendly way to say it is that "Everything is data until it is evaluated.", and you have a lot of control over when evaluation occurs.

For interop, Red has a system-level dialect called Red/System. It's a C level language and is a dialect of Red. That is, Red is high level and is used to define the Red/System dialect, which is used to implement Red. It's the circle of life. :^) You can also compile Red as a library and call into it via an API, so you can use it as an embedded language, or doing things like the Excel example in this blog entry about Red's macros: https://www.red-lang.org/2017/03/062-libred-and-macros.html which also mentions one of the easiest ways to preprocess input, with the `system/lexer/pre-load` feature.

ASTs are mentioned in a few comments, so I should add that while you can certainly do that with Red, it's another thing that can often be avoided entirely. With other langs and tools, you almost have to take that approach, to make it manageable. With Red, there is a function called `parse` which consumes input and supports BNF-like rules to process it. `Parse` is a Red dialect, and the rules are just data it interprets. Rather than building ASTs (though there are some cool examples, and tree-rewriting systems out there), just interpret the input directly.

If it sounds like I'm against metaprogramming, while being on a team creating a language that supports it deeply, that's not the case. Metaprogramming is a great tool for thinking, but it can also make things much harder to debug and maintain. For real work, use what makes your intent clear, and avoids as much complexity as possible.

I'm currently writing a compiler for deep learning with both AOT (emitting Nim code) and hopefully later JIT (emitting LLVM IR) capabilities complete with SIMD support. I don't see how I could do that in another language.

My biggest successes:

- include a quite maintainable JIT for x86_64 (compared to asmjit and xbyak I don't need to parse or codegen the C++ code): https://github.com/numforge/laser/blob/master/laser/photon_j...

- a DSL for neural network: https://github.com/mratsim/Arraymancer#sequence-classificati...

- a matrix multiplication BLAS written from scratch competitive with OpenBLAS and MKL on select matrix shape (2000x2000) but that can also support int8/int16/int32/int64 and not just float32/float64 thanks to metaprogramming. Expanding to new SIMD architecture (ARM) is very easy:

--> benchmark: https://github.com/numforge/laser/blob/master/benchmarks/gem...

--> Metaprogramming AVX512 support: https://github.com/numforge/laser/blob/master/laser/primitiv...

The most important thing for me for metaprogramming is being able to operate on the AST directly

Just to clarify for the general public (because this is often a source of confusion and misunderstandings): first letters are case-sensitive in Nim, so you can do `var car: Car` (where `car` is a name of the variable, and `Car` is a type).

When it comes to style-insensitivity, I also thought this would be a problem when I discovered Nim, but now—2 years later—there was not even one situation where that would cause a problem. (And if you use different styles of the same name to mean different things in other languages: this will bite you sooner or later)

Nim's standard library is written uniformly in camelCase style, which is what I also accepted for my code (even though I come from snake_cased Python).

So why this even exists? To make easier to use libraries written (in another language, e.g. C) in another style, while keeping the uniform style in your code. Basically, this feature makes it easier to use just one style consistently.

To conclude:

If this is the only (or just a major) thing that keeps you from using Nim: I would suggest you to reconsider and give it a chance.

Searching for names is not a problem as people don't mix different styles within the same project. Also there's nimgrep for this.

Finally, getting a compiler error when I try to use variables called should_run and shouldrun in the same scope is a feature, not a bug: it encourage using less misleading names.

Also, it should never consider hell_owl and hello_w_l to be the same, although each could match hellOwl and helloWL. Basically it takes the style-insensitivity too far.

There are style guides and nimpretty to help further.

Racket also supports defmacro[2] like Common Lisp, and it's actually implemented using syntax-case.

I would argue that syntax-case is at least as powerful as defmacro.

[1] https://docs.racket-lang.org/guide/syntax-case.html

[2] https://docs.racket-lang.org/compatibility/defmacro.html

Although your phrasing ('expected/future adoption') makes it sound like you value picking a language that will have loads of jobs available. Clojure has a few solid niches (same number of users as Kotlin IIRC) and is 'symbiotic' with JVM and JS environments. Rust is growing and has supposedly 'safe' BEAM NIF support. Elixir is popular with the Ruby crowd. Although I have to say, 'meta-programming' & 'functional programming' isn't exactly the hottest job market filter.

[0] https://common-lisp.net/implementations

- https://scalameta.org/

- https://speakerdeck.com/itakeshi/metaprogramming-in-scala-th...

- https://geirsson.com/post/2016/02/scalameta/

- https://github.com/milessabin/shapeless

- https://vimeo.com/217863345

As for which language to use to explore these paradigms, the answer would have to be a modern Lisp, i.e. Clojure or Racket. Of those two Clojure is more about being a "pure functional" language and Racket more about meta-programming... in fact Racket has been called a "meta-language".

My `loop for (symbol value) on bindings by #'cddr` and "push onto list in a loop, nreverse and return at the end" macros beg to disagree.

Metaprogramming is just like any other programming, except that the output is fed to the compiler. The way Lisp is compiled, you can even use code generated by code in code that generates other code.

That said, functional programming does indeed fit well with most of the stuff you do when writing code that writes code.

   gensym

Usual suspect here is any kind of "support" code, like serialization, RPC, dependency injection and such.

Comparing Nim's metaprogramming to Rust's shows what I mean. In Rust you can write macros that act on token streams, so as long as the token stream is valid it can be used. This allows libraries like the typed-html package[1].

In Nim you cannot do this unless you put the HTML in a string literal. This limitation created a syntax that is far more idiomatic, admittedly at the expense of familiarity for users of HTML:

    buildHTML():
      html():
        head():
          title: text "My webpage"
      body:
        p(class="red"): text "Hello World"

Please don't let style insensitivity discourage you from using this wonderful language.

1 - https://docs.rs/typed-html/0.2.0/typed_html/

2 - https://github.com/nim-lang/nimforum/blob/master/src/fronten...

There are still plenty of people who dislike Python because "significant whitespace, bleh" with no objective reason why they dislike it except "it gets messed up when I copy the code". That's a tooling problem, just like with style insensitivity, it's not hard to search in a style insensitive manner and in fact all tools should allow this mode for convenience.

  m_?[Yy]_?[Vv]_?[Aa]_?[Rr]_?[Ii]_?[Aa]_?[Bb]_?[Ll]_?[Ee]

See https://haxe.org/manual/macro-ExprOf.html

For example, this seemingly ill-typed definition (which should return Int as suggested by the type paramater, but in fact returns a string) successfully typechecks:

  class StringGetter {
    public static macro function getString():haxe.macro.Expr.ExprOf<Int> {
      var s:String = "";
      return macro $v{s};
    }
  }

  class StringGetter {
    public static macro function getString() {
      var s:String = "";
      return macro ($v{s} : Int);
    }
  }

Expressions generated by your macros will be parsed by the type checked. You won't be able to do

    var s:Int = StringGetter.getString();

That is a weird statement. Haxe expression macros by definition return untyped AST to be processed further normally by the typer, just like hand-written code. And it will be type-checked. And then there's an AST node existing exactly for type-checking (or, rather, type unification) that you can generate to insert custom checks: https://haxe.org/manual/expression-type-check.html

Do use: Common Lisp, Racket, Python, Ruby

Would like to use: Julia, Rust, Elixir

I metaprogram when I see I have already built a mini-language/pattern in my code. Even when writing Lisp, I begin with simple code without extending the language, and only when I have written a bit do I consider looking for patterns.

Yes, but then again, Rust, Julia, and Lisp are _high_ standards of metaprogramming. Python's approach to metaprogramming, like most of its other parts, is simple and concise. For simple tasks that are repeated often, Python sometimes provides easy ways to abstract the how and show only the what. While Python does have metaclasses — which, unfortunately or otherwise, I've never needed to write — I've sometimes found myself reaching for simpler tools like decorators, magic dunder methods, even iterators and generators. And, of course, there's always the ability to generate Python AST, or have something generate it for you.

Examples:

1. the new-in-3.7 feature: dataclasses[0];

2. the library that inspired it: attrs[1];

3. the (de)serialisation library with a strikingly simple design: marshmallow[2];

4. the leaky-abstraction of SQL that makes it somewhat composable: sqlalchemy[3];

5. Google's python bindings for Protocol Buffers[4]; and, of course

6. the lisp-in-Python: Hy[5];

----

[0]: https://www.python.org/dev/peps/pep-0557/

[1]: http://www.attrs.org/

[2]: https://marshmallow.readthedocs.io/en/3.0/_modules/marshmall...

[3]: https://docs.sqlalchemy.org/en/13/orm/extensions/declarative...

[4]: https://developers.google.com/protocol-buffers/docs/pythontu...

[5]: http://docs.hylang.org/en/stable/language/internals.html

[1] https://github.com/darkwiiplayer/moonxml

[2] http://terralang.org/

Although it is certainly not as pure as the Lisp-like languages, it can be an incredibly fast and easy way to create a DSL to solve your problems in a very idiomatic way.

what issues do you have with ruby? it's the de-facto standard for meta-programming

I often use things like `define_method`, `constants`, `method_missing`, etc in select places.

last fun thing I did was adding `let` and `it` to rails's tests.

https://en.wikipedia.org/wiki/Factor_(programming_language)

https://github.com/factor/factor/

http://andreaferretti.github.io/factor-tutorial/ - Good introduction and Metaprogramming section.

You can find good examples of its use by searching for reverse dependencies of template-haskell package[0].

One of these is my tiny experiment that features program synthesis by given type[1]. Recently I added an ability to handle sum types and product types (a.k.a. Either and tuples) -- see 'rewrite' branch.

[0] https://packdeps.haskellers.com/reverse/template-haskell

[1] https://github.com/8084/haskell-holes-th

I know its hated passionately, but I learned about metaprogramming from it.

Also, imo, Racket is the best language for metaprogramming.

Its quite powerful for certain types of problem domains like expert systems.

Compile and run-time MP and quasiquoting are basic features; I'd argue that a language without these doesn't even qualify as "supporting metaprogramming".