Marcos Dione: ayrton-an-update

«So, by simply copying that one and ast.py, modifying the latter so it uses my modified syntax checker, and modifying the setup.py file, I should be done». Famous last words.

I was completely wrong. The _ast module imported by ast.py is not implemented in Python/ast.c but in Python/Python-ast.c. Understanding this, I reread this post I linked to last time and finally[1] understood that changing the syntax would not be so easy. Another eye-opener was the stack trace I got when ast.c's ast_for_call() (the function I'm planning to change) is called when a function call is found in the source. Here, have a look:

#0  ast_for_call (c=0x7fffffffa300, n=0x7ffff700d918, func=0x9ccf40) at Python/ast.c:2470
#1  0x00000000005bfe73 in ast_for_trailer (c=0x7fffffffa300, n=0x7ffff700db98, left_expr=0x9ccf40) at Python/ast.c:2105
#2  0x00000000005c03c0 in ast_for_power (c=0x7fffffffa300, n=0x7ffff6ca4238) at Python/ast.c:2217
#3  0x00000000005c0b01 in ast_for_expr (c=0x7fffffffa300, n=0x7ffff6ca4238) at Python/ast.c:2406
#4  0x00000000005c1108 in ast_for_testlist (c=0x7fffffffa300, n=0x7ffff6ca4030) at Python/ast.c:2562
#5  0x00000000005c118e in ast_for_expr_stmt (c=0x7fffffffa300, n=0x7ffff6facfa8) at Python/ast.c:2584
#6  0x00000000005c3e84 in ast_for_stmt (c=0x7fffffffa300, n=0x7ffff6facfa8) at Python/ast.c:3586
#7  0x00000000005bc94c in <span class="createlink">PyAST FromNodeObject</span> (n=0x7ffff6face18, flags=0x7fffffffa4e0, filename=0x7ffff6d56370, arena=0x8c8780) at Python/ast.c:709
#8  0x000000000050c9e8 in <span class="createlink">PyParser ASTFromStringObject</span> (s=0x7ffff6ca37a0 "foo (a=3, b)", filename=0x7ffff6d56370, start=257, flags=0x7fffffffa4e0, arena=0x8c8780) at Python/pythonrun.c:2160
#9  0x000000000050c6be in <span class="createlink">Py CompileStringObject</span> (str=0x7ffff6ca37a0 "foo (a=3, b)", filename=0x7ffff6d56370, start=257, flags=0x7fffffffa4e0, optimize=-1) at Python/pythonrun.c:2068
#10 0x00000000004c6e54 in builtin_compile (self=0x7ffff70f9c28, args=0x7ffff6ca04f8, kwds=0x0) at Python/bltinmodule.c:671

The compile() builtin (#10) passes through a couple of obscure functions till it reaches the AST build/syntax check code (all the ast_for_*() functions), and we quickly reach it from ast.parse():

def parse(source, filename='<unknown>', mode='exec'):
    return compile(source, filename, mode, PyCF_ONLY_AST)

In a desperate move, I though of hacking a modified version of just that function and do some LD_PRELOAD jedi moves, but the Python interpreter binary (/usr/bin/python3) is statically linked against libpython3. Future versions[2] of the Python interpreter main() function[3] will be really simple:

/* Minimal main program -- everything is loaded from the library */

#include "Python.h"

#ifdef __FreeBSD__
#include <floatingpoint.h>
#endif

int
main(int argc, char **argv)
{
        /* 754 requires that FP exceptions run in "no stop" mode by default,
         * and until C vendors implement C99's ways to control FP exceptions,
         * Python requires non-stop mode.  Alas, some platforms enable FP
         * exceptions by default.  Here we disable them.
         */
#ifdef __FreeBSD__
        fp_except_t m;

        m = fpgetmask();
        fpsetmask(m & ~FP_X_OFL);
#endif
        return Py_Main(argc, argv);
}

So it's not so crazy to think of creating a new interpreter that uses my version of ast_for_call(), but I haven't found the linking command that makes that possible. So this part of the project is in the freezer.

Meanwhile, I spent a couple of hours with the code and managed to implement piping and redirection, even with some tests! You can have an idea of how it works reading the README[4]. Also, expect a release soon.

As for replacing sh, no efforts (besides burning some glucose and firing some neurons) have been done.


[1] Also famous last words.

[2] Python-3.3 has a more complex version which has to handle making two copies of argv while dealing with the system's encoding.

[3] It's weird, the source code of the executable resides in Modules/python.c. Go figure.

[4] Some day I'll convert that file to reST, make it the tutorial, and have it with the rest of the documentation.


python ayrton

Marcos Dione: near-future-of-ayrton

It might look like ayrton's development has stalled. First, yes, a little. Last week I barely put any time on it. Second, development will be less release centered, and only because I have three things in mind that are very interesing to me that I want to implement, and it seems that more or less I will be doing them on a «I want to hack on this right now» basis.

One is to replace sh. So far I have been tiptoeing around it to make output go to the terminal by default, which forces the intoroduction of Capture, and to not raise an exception if the command fails. But recently I tried to make an ayrton script to more or less automatize releases and I found out that sh really messes with the subprocess' view of the terminal. In short, editors are not usable if they're launched from sh. I could not say that this is the last straw; I still think that sh is very good at what it does, but this definetely is like a sequoia trunk after a few straws. This is called issue #15.

Another is to try to make several changes on the way output is captured, piped and redirected. I find this thing of messing with _out too annoying. Piping could be handled by |, but currently is not possible to do it. Redirections could be handled with >, but as long as I don't have |, it doesn't make much sense. I'm already playing with checking the AST produced by the source, because I'm trying to minimize the situations in which unknown names are resolved into executables, and these things also require AST handling, so it should be more or less easy to achieve.

There is another, more ambitious thing. So far I've been using Python3's syntax, but this imposes some restrictions. I like sh's keyword arguments, but the syntax doesn't allow you to put positional arguments after keyword arguments, like:

rsync (archive=True, src, dst)

So the goal is to use another parser. As this is the only modification in mind, it only makes sense to take Python3's parser, modify what I want, and ship with that. But the question is, where is the parser code?

The answer is not simple, but at least exists besides, the source code. I mean, I tried to untangle how this works, but as Eli Bendersky shows in this post, it is not a simple thing. Luckily, what I plan to change is not implemented in the parser itself, but in the syntax checker, which is implemented in the file ast.c. So, by simply copying that one and ast.py, modifying the latter so it uses my modified syntax checker, and modifying the setup.py file, I should be done. Let's see how it goes...


python ayrton

Joaquin Sorianello: Chau Carlitos

La semana pasada falleció mi querido abuelo Carlos, un inmenso filosofo y guia en el trajín de la existencia.

Sentí la necesidad de escribir unas palabras para leerle a la familia, antes de que su cuerpo sea cenizas. Lo comparto aquí:


Pensé que sabía como era la tristeza.
Pero no sabia.
En este ultimo tiempo, empece sin darme cuenta a vivir como vivías: mas despacio, mas en el presente; saboreando el tiempo sin apuro, disfrutando toda la belleza sutil que los hombres pudieron ver en un tiempo. Y es tan bello.
Que suerte tuvimos de tenerte cerca, paisano del medioevo. Nos dejaste las herramientas y la sabiduría para transformarte en leyenda. Nos dejaste el amor por la simpleza, por lo bello, por los culos de rebosantes gorditas porteñas.
Nos querias. Lo demostrabas con formas acordes a tu personalidad.Nunca regalaste un libro. Regalaste invitaciones a leerlos, y cumpliste tu objetivo de despertar miradas y enseñar a ver.
Ahora la digestora del tiempo nos hará recordarte. Nos la dejaste fácil. Basta con repetir tus frases, escuchar tus canciones favoritas. Mirar con ojos atentos, o sentarse en el banquito (del museo de Bellas Artes) a planear un robo romantico e imposible. Como vos.
Vas a ser el héroe de tataranietos curiosos por saber de ese pequeño cúmulo de sabiduría, amor y sensibilidad. Gracias. Por escuchar y por ser. Por dejarnos un  cacho tuyo aun por descubrir.
Ha muerto un gran hombre. Lloro por los irreversible.

Marcos Dione: ayrton-0.2

It's that time of the year already: a new ayrton version. This time I got more improvements than before, and for the first time[1] I introduce a backwards incompatible change, but at least is well Documented in the Debian-like NEWS.rst file, and it's for a good cause[2].

So, without further addo, the ChangeLog (mixed with the NEWS)[3]:

  • New function options() is similar to bash's set command. So far only the errexit and its short versions is accepted.
  • The ssh() context manager was renamed to remote() so the ssh executable is stills reachable from code. This was due to the fact that ssh is too complex to mimic.
  • New function shift() similar to bash's command of the same name. See the docs.
  • New function source() similar to bash's command[4] allows to "import" another script's local definitions into the "sourcing" script's.

As usual, you can grab the sources from pypi or GitHub.


[1] No, really, who can actually come up with that phrases when the product has only 3 previous releases... take it with a grain of salt.

[2] That cause being that I'm lazy and I don't plan to reimplement all the functionality available in the ssh executable.

[3] Actually, this changelog is proof-read nad more complete that any other that you will find elsewhere. Code burns in the disk and I'm not so used to this release thing yet. I hope to get better soon.

[4] Do you see the pattern here?


python ayrton

Gustavo Campanelli: Eve Online: Ayudas externas

Una de las particularidades del Eve Online es el grado de respeto que tiene CCP, la compañía, con sus usuarios. A la vez, el juego pose estructura y reglas, pero además de eso es un gran sandbox (arenenro) donde cada cual inveta su juego. Esto se traduce en una base de usuarios tremendamente fiel que además de generar eventos dentro del juego, construye cosas alrededor del juego, afuera del juego

Gustavo Campanelli: Los mecanismos predictores

Vengo a confesar algo: no me gusta la mayoría de los mecanismos predictores. Para aclarar, un mecanismos predictor es aquel que mediante algún algoritmo intenta predecir lo que vamos a precisar a continuación. Realmente me parecen muy feos en los smartphones, donde son una ayuda porque el sistema de entrada de datos de esos teléfono es realmente feo. Sea esto debido a que el teclado es muy

Gustavo Campanelli: Eve Online: mercado y actualizaciones de juego

Si bien este artículo habla principalmente del juego Eve Online, en realidad es aplicable a varios juegos distintos, todos aquellos juegos que además de poseer mercado tiene un sistema de construcción de items. Los mercados donde los jugadores son los que proponen los precios son variables en el tiempo, esto quiere decir que un mineral que hoy me cuesta 4,50 la unidad (moneda del juego) puede