Damián Avila: My IPython-powered semi-automatic git workflow

This is the last post of this year, so I try to do my best to give you something interesting to think about...

In this case, I will show you my git workflow... and you know there are a lot of workflows out there... and probably better than mine, but I just want to share with you the place where I find myself comfortable.

And yes... my git workflow is also powered by IPython (I am very repetitive when I love a project!). And it is a semi-automatic one, using the IPython notebooks (ipynbs) as a sort of templates, transforming them into a new conceptual entity: the ipytmpl (and yes, I love to invent names too!).

Read more… (10 min remaining to read)

Alberto Paparelli: Incursionando en los embutidos

Una de las tantas cosas que me gusta hacer como pasatiempo es cocinar, y hace mucho tiempo que tenia ganas de empezar a probar con los fiambres caseros.

Busque un poco en Internet, y encontré la bondiola, la cual parece ser uno de los embutidos más fáciles para hacer.

La hice, y salio muy bien, Así que ahora tengo ganas de hacer otros embutidos como Jamón Crudo y Quesos, y obviamente, otra bondiola, porque ya la comimos.

Así quedo la bondiolita casera.

Bondiolita casera

Damián Avila: A 'poor man' spell checker for the IPython notebook

OK, today I will release another IPython js extension: Spellchecker, which obviously do what you are thinking... spell check the content of your IPython notebook cells.

And why it is a poor man extension? Because it is a simple workaround to get the spell checker functionality and not a broad solution... but it works, and solve my spelling problems!

Read more… (4 min remaining to read)

Hernán Grecco: Context aware unit conversion in Pint

Today I am releasing version 0.4 of Pint, a Python units library.

Pint is Python package to define, operate and manipulate physical quantities: the product of a numerical value and a unit of measurement. It allows arithmetic operations between them and conversions from and to different units.

It provides a comprehensive and extensive list of physical units, prefixes and constants defined in a standalone text file. The registry can parse prefixed and pluralized forms of units resulting in a much shorter and maintainable unit definition list.

It also provides great NumPy integration, with implicit unit conversion and an emphasis on correctness.

What's new

Pint 0.4 introduces the concept of Context. A Context enables to convert between unrelated dimensions based on pre-established rules. For example, in spectroscopy you might require to convert between wavelength and frequency. As expected, Pint will raise an error if you try to do this:

    >>> import pint
    >>> ureg = pint.UnitRegistry()
    >>> q = 500 * ureg.nm
    >>> q.to('Hz')
    Traceback (most recent call last):
    pint.unit.DimensionalityError: Cannot convert
    from 'nanometer' ([length]) to 'hertz' (1 / [time])

But if you enable the spectroscopy context:

    >>> q.to('Hz', 'spectroscopy')
    <Quantity(5.99584916e+14, 'hertz')>

the relation `frequency = speed_of_light / wavelength` is used.
If you have multiple transformations, you can use the `with` statement to enable a particular context for a block of code:

    >>> with ureg.context('spectroscopy'):
    ...     q1.to('Hz')
    ...     q2.to('Hz')

For convenience, most contexts have a short version of the name:

    >>> q.to('Hz', 'sp')
    <Quantity(5.99584916e+14, 'hertz')>

And there is more for contexts: enabling a context for all operations, contexts that take parameters, defining your own context in a definition file or programatically. Take a look at the context documentation for more information.

There is one backwards incompatible change in this version. In previous versions of Pint, comparing two quantities containing NumPy Arrays resulted in a boolean:

        >>> np.ones(3) * ureg.meter == np.ones(3) * ureg.meter

This output was convenient in some cases but unexpected if you have worked with NumPy. To improve the integration in scientific python packages, the output is now consistent with NumPy:

        >>> np.ones(3) * ureg.meter == np.ones(3) * ureg.meter
        array([ True,  True,  True], dtype=bool)

To recover the previous result, you need to use the usual NumPy all function:

       >>> np.all(np.ones(3) * ureg.meter == np.ones(3) * ureg.meter)

Thanks to the people that contributed bug reports, suggestions and patches since 0.3. In particular to: John David Reaver, Giel van Schijndel and Nate Bogdanowicz.

Interested? Install it and give it a try!

Submit your bug reports, comments and suggestions in the Issue Tracker. There are already some ideas for version 0.5. Check them out, comment and add yours.

Read the docs: https://pint.readthedocs.org/
or fork the code: https://github.com/hgrecco/pint

Diego Sarmentero: Mis charlas en PyCon Uruguay 2013

Este año tuve la oportunidad de asistir y dar 2 charlas y una lightning talk en la PyCon de Uruguay, la verdad el evento me gusto muchisimo!! La gente muy buena onda, y el lugar y la organización me parecieron geniales.

Espero poder volver a ir el año que viene, porque fue una muy buena experiencia!

Aca les dejo los videos y material de las charlas que di (que ahora me doy cuenta que me habia olvidado de hacer un post al respecto):

Introducción a PyQt

Lightning Talk: Documentor

NINJA-IDE, un IDE especialmente diseñado para Python

Material de las charlas:

Y aca dejo algunas fotos (muy pocas) que saque durante el evento: PyCon Uruguay 2013

Damián Avila: Don't write scripts, just write ipyscripts!

I usually have some repetitive tasks in my daily work-flow. You probably have some of those too. To save some minutes from your day, you probably write little scripts to do these jobs.

I used to do that... but now I am only writing little IPython notebooks to do these simple jobs, I call them ipyscripts (yes, I won a prize for originality, he he!).

These ipyscripts are IPython-powered, so we can use all the IPython's machinery to do complex things in a simple way...

Read more… (3 min remaining to read)

Marcos Dione: restoring-files-times

Do you remember this? Well, now I have a similar but quite different case: I want to restore the files' times. Why? Read me out.

Since a long time I'm doing backups to an external drive. For that I simply use rsync. When I bought my second laptop, I took the chance to test restoring from that backup, so I just did a last backup in the old machine and a restore in the new one. This allowed me to add the last few things that were missing. After a few more weeks finding missing things, doing more backup/restore cycles between the old and new laptops, I was happy.

But there was something that I didn't realize till a few months later. The file times were not backed up, and clearly not restored. This didn't affect me in any way, at least until I tried to a new post to this glob.

See, this glob is generated with ikiwiki. ikiwiki uses the files' mtime to generate the posts' times, and uses that to sort them and to assign them as belonging to some year or month. With the mtimes lost, ikiwiki was assigning all the older posts to some day in Oct, 2012, which was the date of the last backup/restore cycle.

Frustrated (yes, sometimes I have a low threshold), I left it like that: no planet was flooded because in any case the rss feeds didn't really change, and nobody would notice and complain about it. In fact, nobody did.[1] Have in mid that as I kept my old laptop as a server, and that I didn't clean up all the files I keep in my backup, I still had the original mtimes in the original files.

Yesterday, playing around with unrelated things, I came across the post I mentioned at the beginning of this one, and had the urge to really fix that mess. Today I started looking at it, and found that this command was some part of it:

find src/projects/glob/work/ -name *.mdwn | xargs stat --format '%X %Y %Z %n'

but processing its output in bash seemed like a burden. Luckily, I have the solution for that, a project that I started for exactly that same reason: bash sucks at manipulating data.

So I grabbed my editor, punched keys for some 7 minutes, run 3 or 4 tests, making sure that everything would go smooth, run once more on the live system, rebuilt my glob[2] and voilà! It's fixed now.

How long is the code? Not much really:

from os import utime

with remote ('', allow_agent=False, password='XXXXXXXX') as fds:
    # find al files (not only posts), get its times and path
    with cd ('src/projects/glob/work'):
        find ('.', '-name', '*.mdwn') | xargs ('stat', format='%X %Y %Z %n')

(i, o, e)= fds

for line in e.readlines ():
    print (line.decode ('utf-8'), end='')

for line in o.readlines ():
    data= line.split ()
    access, modif, change= [ int (x) for x in data[:3] ]
    file_path= data[3].decode ('utf-8')
    if _e (file_path):
        utime (file_path, times= (access, modif))
        print ("%s not found!" % file_path)

Notice that this example is even more complex than initially thought: it ssh's into the server to fetch the original times.

This makes one thing clear: I have to improve a lot the remote() API. For instance, so far I have no way to know if the remote commands executed fine or not, and the way to read the lines is still ugly and not homogeneous with how other ayrton functions/executables work.

Also note that this does not restore the ctimes, as I hadn't found a way to do it, but didn't really did a lot of effort. I don't really need them.

Of course, I also fixed my backup and restore scripts :)

[1] I was this >< close of adding #foreveralone to that sentence. I think I'll resist for a couple more years.

[2] Don't forget, ikiwiki compiles static sites.

ayrton python sysadmin