Mariano Guerra: Multi-Paxos with riak_ensemble Part 3

In the previous post I showed how to use riak_ensemble in a rebar3 project, now I will show how to create an HTTP API for the Key/Value store using Cowboy and jsone.

This post assumes that you have erlang and rebar3 installed, I'm using erlang 19.3 and rebar3 3.4.3.

The source code for this post is at https://github.com/marianoguerra/cadena check the commits for the steps.

Dependency Setup

To have an HTTP API we will need an HTTP server, in our case we will use Cowboy 2.0 RC 3, for that we need to:

  1. Add it as a dependency (we will load if from git since it's still a release candidate)
  2. Add it to our list of applications to start when our application starts
  3. Add it to the list of dependencies to include in our release
  4. Set up the HTTP listener and routes when our application starts

We setup just one route that is handled by the cadena_h_keys module, it's a plain HTTP handler, no fancy REST stuff for now, there we handle the request on the init/2 function itself, we pattern match against the method field on the request object and handle:

POST
set a key in a given ensemble to the value sent in the JSON request body
GET
get a key in a given ensemble, if not found null will be returned in the value field in the response
DELETE
delete a key in a given ensemble, returns null both if the key existed and if itdidn't

Any other method would get a 405 Method Not Allowed response.

The route has the format /keys/<ensemble>/<key>, for now we only allow the root ensemble to be set in the <ensemble> part of the path.

We also add the jsone library to encode/decode JSON and the lager library to log messages.

We add both to the list of dependencies to include in the release.

We will also need to have a way to override the HTTP port where each instance listens to so we can run a cluster on one computer and each node can listen for HTTP requests on a different port.

The dev and prod releases will listen on 8080 as specified in vars.config.

node1 will listen on port 8081 (override in vars_node1.config)

node2 will listen on port 8082 (override in vars_node2.config)

node3 will listen on port 8083 (override in vars_node3.config)

To avoid having to configure this in sys.config we will define a cuttlefish schema in config.schema that cuttlefish will use to generate a default config file and validation code for us.

We have to replace the variables from variable overrides in our config.schema file for each release before it's processed by cuttlefish itself, for that we use the template directive on an overlay section on the release config.

Build devrel:

make revrel

Check the configuration file generated for each node at:

_build/node1/rel/cadena/etc/cadena.conf
_build/node2/rel/cadena/etc/cadena.conf
_build/node3/rel/cadena/etc/cadena.conf

The first part is of interest to us, it looks like this for node1, the port number is different in node2 and node3:

## port to listen to for HTTP API
##
## Default: 8081
##
## Acceptable values:
##   - an integer
http.port = 8081

## number of acceptors to user for HTTP API
##
## Default: 100
##
## Acceptable values:
##   - an integer
http.acceptors = 100

## folder where ensemble data is stored
##
## Default: ./cadena_data
##
## Acceptable values:
##   - text
data.dir = ./cadena_data

Start 3 nodes in 3 different shells:

make node1-console
make node2-console
make node3-console

Start enseble and join nodes, I created a target called devrel-setup in the Makefile to make it easier:

make devrel-setup

Let's set key1 in ensemble root to 42 on node1 (port 8081):

curl -X POST http://localhost:8081/keys/root/key1 -d 42

Response:

{"data":{"epoch":2,"key":"key1","seq":10,"value":42},"ok":true}

Let's get key1 in ensemble root to 42 on node2 (port 8082):

curl -X GET http://localhost:8082/keys/root/key1

Response:

{"data":{"epoch":2,"key":"key1","seq":10,"value":42},"ok":true}

Same on node3:

curl -X GET http://localhost:8083/keys/root/key1

Response:

{"data":{"epoch":2,"key":"key1","seq":10,"value":42},"ok":true}

Overwrite on node1:

curl -X POST http://localhost:8081/keys/root/key1 -d '{"number": 42}'

Response:

{"data":{"epoch":2,"key":"key1","seq":400,"value":{"number":42}},"ok":true}

Get on node2:

curl -X GET http://localhost:8082/keys/root/key2
{"data":{"epoch":3,"key":"key2","seq":11,"value":null},"ok":true}

Let's set key2 in ensemble root to {"number": 42} on node1 (port 8081):

curl -X POST http://localhost:8081/keys/root/key2 -d '{"number": 42}'

Response:

{"data":{"epoch":3,"key":"key2","seq":67,"value":{"number":42}},"ok":true}

Get it on node2:

curl -X GET http://localhost:8082/keys/root/key2

Response:

{"data":{"epoch":3,"key":"key2","seq":67,"value":{"number":42}},"ok":true}

Delete key2 in ensemble root on node2:

curl -X DELETE http://localhost:8082/keys/root/key2

Response:

{"data":{"epoch":3,"key":"key2","seq":137,"value":null},"ok":true}

Check that it was removed by trying to get it again on node2:

curl -X GET http://localhost:8082/keys/root/key2

Response:

{"data":{"epoch":3,"key":"key2","seq":137,"value":null},"ok":true}

There you go, now you have a Consistent Key Value Store with an HTTP API.

Mariano Guerra: Multi-Paxos with riak_ensemble Part 2

In the previous post I showed how to use riak_ensemble from the interactive shell, now I will show how to use rebar3 to use riak_ensemble from a real project.

This post assumes that you have erlang and rebar3 installed, I'm using erlang 19.3 and rebar3 3.4.3.

The source code for this post is at https://github.com/marianoguerra/cadena check the commits for the steps.

Create Project

rebar3 new app name=cadena
cd cadena

The project structure should look like this:

.
├── LICENSE
├── README.md
├── rebar.config
└── src
        ├── cadena_app.erl
        ├── cadena.app.src
        └── cadena_sup.erl

1 directory, 6 files

Configuring Dev Release

We do the following steps, check the links for comments on what's going on for each step:

  1. Add Dependencies
  2. Configure relx section
    1. Add overlay variables file vars.config
    2. Add sys.config
    3. Add vm.args

Build a release to test that everything is setup correctly:

$ rebar3 release

Run the release interactively with a console:

$ _build/default/rel/cadena/bin/cadena console

Output (edited and paths redacted for clarity):

Exec: erlexec
        -boot _build/default/rel/cadena/releases/0.1.0/cadena
        -boot_var ERTS_LIB_DIR erts-8.3/../lib
        -mode embedded
        -config    _build/default/rel/cadena/generated.conf/app.1.config
        -args_file _build/default/rel/cadena/generated.conf/vm.1.args
        -vm_args   _build/default/rel/cadena/generated.conf/vm.1.args
        -- console

Root: _build/default/rel/cadena
Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:4:4] [async-threads:64]
                      [kernel-poll:true]

18:31:12.150 [info] Application lager started on node 'cadena@127.0.0.1'
18:31:12.151 [info] Application cadena started on node 'cadena@127.0.0.1'
Eshell V8.3  (abort with ^G)
(cadena@127.0.0.1)1>

Quit:

(cadena@127.0.0.1)1> q().
ok

Non interactive start:

$ _build/default/rel/cadena/bin/cadena start

No output is generated if it's started, we can check if it's running by pinging the application:

$ _build/default/rel/cadena/bin/cadena ping

We should get:

pong

If we want we can attach a console to the running system:

$ _build/default/rel/cadena/bin/cadena attach

Output:

Attaching to /tmp/erl_pipes/cadena@127.0.0.1/erlang.pipe.1 (^D to exit)

(cadena@127.0.0.1)1>

If we press Ctrl+d we can dettach the console without stopping the system:

(cadena@127.0.0.1)1> [Quit]

We can stop the system whenever we want issuing the stop command:

$ _build/default/rel/cadena/bin/cadena stop

Output:

ok

Note

Use Ctrl+d to exit, if we write q(). not only we dettach the console but we also stop the system!

Let's try it.

Non interactive start:

$ _build/default/rel/cadena/bin/cadena start

No output is generated if it's started, we can check if it's running by pinging the application:

$ _build/default/rel/cadena/bin/cadena ping

We should get:

pong

If we want we can attach a console to the running system:

$ _build/default/rel/cadena/bin/cadena attach

Output:

Attaching to /tmp/erl_pipes/cadena@127.0.0.1/erlang.pipe.1 (^D to exit)

(cadena@127.0.0.1)1>

Now let's quit with q():

(cadena@127.0.0.1)1> q().

Output:

ok

Now let's see if it's alive:

$ _build/default/rel/cadena/bin/cadena ping

Node 'cadena@127.0.0.1' not responding to pings.

Be careful with how you quit attached consoles in production systems :)

Configure Prod and Dev Cluster Releases

Building Prod Release

We start by adding a new section to rebar.config called profiles, and define 4 profiles that override the default release config with specific values, let's start by trying the prod profile, which we will use to create production releases of the project:

rebar3 as prod release

Output:

===> Verifying dependencies...
...
===> Compiling cadena
===> Running cuttlefish schema generator
===> Starting relx build process ...
===> Resolving OTP Applications from directories:
          _build/prod/lib
          erl-19.3/lib
===> Resolved cadena-0.1.0
===> Including Erts from erl-19.3
===> release successfully created!

Notice now that we have a new folder in the _build directory:

$ ls -1 _build

Output:

default
prod

The results of the commands run "as prod" are stored in the prod folder.

You will notice if you explore the prod/rel/cadena folder that there's a folder called erts-8.3 (the version may differ if you are using a different erlang version), that folder is there because of the include_erts option we overrided in the prod profile.

This means you can zip the _build/prod/rel/cadena folder, upload it to a server that doesn't have erlang installed in it and still run your release there.

This is a good way to be sure that the version running in production is the same you use in development or at build time in your build server.

Just be careful with deploying to an operating system too different to the one you used to create the release becase you may have problems with bindings like libc or openssl.

Running it is done as usual, only the path changes:

_build/prod/rel/cadena/bin/cadena console

_build/prod/rel/cadena/bin/cadena start
_build/prod/rel/cadena/bin/cadena ping
_build/prod/rel/cadena/bin/cadena attach
_build/prod/rel/cadena/bin/cadena stop

Building Dev Cluster Releases

To build a cluster we need at least 3 nodes, that's why the last 3 profiles are node1, node2 and node3, they need to have different node names, for that we use the overlay var files to override the name of each, that is achieved on config/vars_node1.config for node1, config/vars_node2.config for node2 and config/vars_node3.config for node3.

Now let's build them:

rebar3 as node1 release
rebar3 as node2 release
rebar3 as node3 release

The output for each should be similar to the one for the prod release.

Now on three different shells start each node:

./_build/node1/rel/cadena/bin/cadena console

Check the name of the node in the shell:

(node1@127.0.0.1)1>

Do the same for node2 and node3 on different shells:

./_build/node2/rel/cadena/bin/cadena console
./_build/node3/rel/cadena/bin/cadena console

You should get respectively:

(node2@127.0.0.1)1>

And:

(node3@127.0.0.1)1>

In case you don't remember, you can quit with q().

Joining the Cluster Together

Until here we built 3 releases of the same code with slight modifications to allow running a cluster on one computer, but 3 nodes running doesn't mean we have a cluster, for that we need to use what we learned in the Multi-Paxos with riak_ensemble Part 1 but now on code and not interactively.

For that we will create a cadena_console module that we will use to make calls from the outside and trigger actions on each node, the code is similar to the one presented in Multi-Paxos with riak_ensemble Part 1.

join([NodeStr]) ->
    % node name comes as a list string, we need it as an atom
    Node = list_to_atom(NodeStr),
    % check that the node exists and is alive
    case net_adm:ping(Node) of
        % if not, return an error
        pang ->
            {error, not_reachable};
        % if it replies, let's join him passing our node reference
        pong ->
            riak_ensemble_manager:join(Node, node())
    end.

create([]) ->
    % enable riak_ensemble_manager
    riak_ensemble_manager:enable(),
    % wait until it stabilizes
    wait_stable().

cluster_status() ->
    case riak_ensemble_manager:enabled() of
        false ->
            {error, not_enabled};
        true ->
            Nodes = lists:sort(riak_ensemble_manager:cluster()),
            io:format("Nodes in cluster: ~p~n",[Nodes]),
            LeaderNode = node(riak_ensemble_manager:get_leader_pid(root)),
            io:format("Leader: ~p~n",[LeaderNode])
    end.

We also need to add the riak_ensemble supervisor to our supervisor tree in cadena_sup:

init([]) ->
    % get the configuration from sys.config
    DataRoot = application:get_env(riak_ensemble, data_root, "./data"),
    % create a unique path for each node to avoid clashes if running more
    % than one node in the same computer
    NodeDataDir = filename:join(DataRoot, atom_to_list(node())),

    Ensemble = {riak_ensemble_sup,
                {riak_ensemble_sup, start_link,
                 [NodeDataDir]},
                permanent, 20000, supervisor, [riak_ensemble_sup]},

    {ok, { {one_for_all, 0, 1}, [Ensemble]} }.

Before building the dev cluster we need to add the crypto app to cadena.app.src since it's needed by riak_ensemble to create the cluster.

Now let's build the dev cluster, I created a Makefile to make it simpler:

make devrel

On three different shells run one command on each:

make node1-console
make node2-console
make node3-console

Let's make an rpc call to enable the riak_ensemble cluster on node1:

./_build/node1/rel/cadena/bin/cadena rpc cadena_console create

On node1 you should see something like:

[info] {root,'node1@127.0.0.1'}: Leading

Let's join node2 to node1:

./_build/node2/rel/cadena/bin/cadena rpc cadena_console join node1@127.0.0.1

On node1 you should see:

[info] join(Vsn): {1,152} :: 'node2@127.0.0.1' :: ['node1@127.0.0.1']

On node2:

[info] JOIN: success

Finally let's join node3:

./_build/node3/rel/cadena/bin/cadena rpc cadena_console join node1@127.0.0.1

Output on node1:

[info] join(Vsn): {1,453} :: 'node3@127.0.0.1' :: ['node1@127.0.0.1','node2@127.0.0.1']

On node3:

[info] JOIN: success

Let's check that the 3 nodes have the same view of the cluster, let's ask node1 what's the ensemble status:

./_build/node1/rel/cadena/bin/cadena rpc cadena_console ensemble_status
Nodes in cluster: ['node1@127.0.0.1','node2@127.0.0.1','node3@127.0.0.1']
Leader: 'node1@127.0.0.1'

node2:

$ ./_build/node2/rel/cadena/bin/cadena rpc cadena_console ensemble_status
Nodes in cluster: ['node1@127.0.0.1','node2@127.0.0.1','node3@127.0.0.1']
Leader: 'node1@127.0.0.1'

node3:

$ ./_build/node3/rel/cadena/bin/cadena rpc cadena_console ensemble_status
Nodes in cluster: ['node1@127.0.0.1','node2@127.0.0.1','node3@127.0.0.1']
Leader: 'node1@127.0.0.1'

Everything looks right, stop the 3 nodes (q().) and start them again, you will see that after starting up node1 logs:

[info] {root,'node1@127.0.0.1'}: Leading

And if you call ensemble_status on any node you get the same outputs as before, this means they remember the cluster topology even after restarts.

Juanjo Conti: Goodreads review: Los gauchos irónicos (Juan Terranova)

Lo cargué en mi celular y lo fui leyendo en diferentes momentos. Me gustó el tipo de análisis de los textos, yendo al detalle, al fondo. Mis reseñas/críticas/artículos preferidos fueron:

* Lamberti
* Catálogo animal y altura en Busqued
* Actividad paranormal, Mazinger Z y terrorismo de Estado
* Sobre la trilogía argentina de Pablo Katchadjian
* Sobre 76 de Félix Bruzzone
* Internet y literatura

Algunas ideas que marqué en los últimos dos textos:

Sobre no citar:

En Fogwill vemos otra forma de juventud, la que no necesita citar autores prestigiosos y consensuados, la que no se arrodilla ante el conocimiento, la que no desea pertenecer a un sistema si no es en sus propios términos. Es la juventud que señala y ríe. La juventud que pretende, al mismo tiempo, ser fiscal y superación de todo. La juventud de la picaresca, el arrebato, la paranoia y las posiciones antifóbicas.

Sobre los blogs:

La gran contradicción de un diario privado que es público no resulta tan dura como que la tecnología comienza a crear y a confirmar autores. La primera persona arrecia. La subjetividad se inflama.

Sobre el chat:

Cuando se describió por primera vez el chat se dijo “un sistema de mensajes instantáneos”. Pero, ¿más instantáneos que el mail? Sí. El protocolo de comunicación es diferente. Las presiones y operaciones que ejerce el chat sobre la lengua son más duras y exigentes que en el mail. En el chat aparecen aun con más contundencia y claridad los vectores de condensación y oralidad. Podríamos incluso decir que aparecen con brutalidad. Los usuarios retuercen la lengua.


Rating: 3/5

Original: https://www.goodreads.com/review/show/2039923905

Juanjo Conti: Goodreads review: La casa de los eucaliptus (Luciano Lamberti)

Francisco me prestó el libro para que lea el cuento La casa de los eucaliptus porque estaba relacionado (en estilo, no en contenido) con una nouvelle que estamos revisando. Ya que estaba leí todos:

* Los caminos internos: es un cuento corto, está bueno para abrir el libro. Es un cover de La tercera expedición de Crónicas marcianas de Ray Bradbury. Hay problema en esto? No. Mejor lo explica Saer en su ensayo El largo adiós a propósito del género policial: "“admitida una serie de convenciones narrativas, el valor de una obra puede residir en la utilización de esas convenciones para aportar resultados poéticos desde su interior".

* La casa de los eucaliptus: el mejor del libro. Es un cuento largo? Es una nouvelle? La discusión no sirve más que para el onanismo intelectual, pero, ya que estoy escribiendo, voy a tomar partido por la primera opción ya que se estructura más entorno a una acción (los asesinatos) que a un personaje (el asesino).

* El tío Gabriel: este no me gustó. El hecho fantástico que se describe está demasiado fuera de lugar. Tanto que no es un relato de terror ni fantástico, es un relato de humor y me parece que desentona en el libro.

* Los chicos de la noche: venía bien pero tampoco me terminó de cerrar. Puede ser que sea porque se me escapa la referencia a la novela Demian. El final me resultó apresurado.

* El Espíritu Eterno: me gustó mucho como está escrito, me pareció original. Sin haberla ido a ver al cine aún, me recordó la película La cordillera. El final no me gustó. Por un lado aparece un elemento que me hizo ruido (que no encajó en la idea que me venía haciendo del cuento) y por el otro no me dejó satisfecho el recurso de un "secreto" que no se le revela al lector.

* Vida de E.: otro cuento que leo como un cover. En este caso de los cuentos biográficos de Bolaño (como Buba). Pero en este caso el cover está más bueno que el tema original. Me gustó mucho como se introduce el elemento fantástico en la trama. Mi segundo cuento preferido del libro.

* La ventana: no tengo nada que decir de este cuento.

* Eddie: lo salteé porque ya lo había leído online. Me recordó el tema John El Esquizofrénico de Calle 13.

* Muñeca: creo que es el cuento más tenebroso del libro. Y ya que estoy comparando, Muñeca se me hace una versión de La gallina degollada de Horacio Quiroga pero mucho más terrible porque la víctima de los idiotas sigue viva, sufriendo todos los días.

* Acapulco: está bueno. Tiene algo de IT, algo de The body. El cuento más Stephen King del libro. El orden de las partes me confundió un poco.

* Carolina baila y Santa: los dos últimos cuento me parecieron (por razones distintas) los más originales. El primero por la forma de introducir un elemento moderno: Facebook. El segundo por la forma que tomó el cuento: los comentarios de un cronista de una crónica que no leímos.

PD: mis teorías sobre cuentos que son covers de otro habla más de mis (acotadas) lecturas que de las intenciones del autor.

Rating: 3/5

Original: https://www.goodreads.com/review/show/2117167111

Mariano Guerra: Public/Private Key Encryption, Sign and Verification in Erlang

You want to encrypt/decrypt some content?

You want to generate a signature and let others verify it?

At least that's what I wanted to do, so here it is.

First generate keys if you don't have some available:

openssl genrsa -out private.pem 2048
openssl rsa -in private.pem -out public.pem -outform PEM -pubout

Load the raw keys:

{ok, RawSKey} = file:read_file("private.pem").
{ok, RawPKey} = file:read_file("public.pem").

[EncSKey] = public_key:pem_decode(RawSKey).
SKey = public_key:pem_entry_decode(EncSKey).

[EncPKey] = public_key:pem_decode(RawPKey).
PKey = public_key:pem_entry_decode(EncPKey).

Let's encrypt a message with the private key and decrypt with the public key:

Msg = <<"hello crypto world">>.
CMsg = public_key:encrypt_private(Msg, SKey).
Msg = public_key:decrypt_public(CMsg, PKey).

We can do it the other way, encrypt with the public key and decrypt with the private key:

CPMsg = public_key:encrypt_public(Msg, PKey).
Msg = public_key:decrypt_private(CPMsg, SKey).

Let's generate a signature for the message that others can verify with our public key:

Signature = public_key:sign(Msg, sha256, SKey).
public_key:verify(Msg, sha256, Signature, PKey).

% let's see if it works with another message
public_key:verify(<<"not the original message">>, sha256, Signature, PKey).

Juanjo Conti: Goodreads review: A medio borrar (Juan José Saer)

Había colgado el libro pocas páginas antes de terminar el último texto que lo compone, la nouvelle (creo que en la publicación, La mayor, original se presenta como cuento), que también le da título al libro, A medio borrar. En este relato, se cuenta la historia de cuando Pichón Garay se va de Santa Fe para instalarse a vivir en París.

En el comienzo del libro se destaca el muy rico prólogo de Martín Prieto, en donde se dice (hablando del programa de Saer): "desactivar el modelo de los relatos cerrados, con principio desarrollo y fin y su consecuente ilusión de totalidad". Más adelante, sobre su singular sintaxis, distinguible por "la extensión de las frases y el uso de los signos de puntuación, un tipo de descripción morosa pero siempre narrativa, es decir, siempre puesta en función del relato y no del mero preciosismo".

De la primera parte, Argumentos, destaco: En la costra reseca, Al abrigo (para mí, uno de sus mejores cuentos) y de la segunda parte, Cuentos, destaco Palo y hueso.

Al final de cada texto hay actividades para el aula. Veremos si se hace uso de ellas en las escuelas de la provincia. Algunos textos de los seleccionados me parecieron un poco difíciles como entrada al autor.

Rating: 4/5

Original: https://www.goodreads.com/review/show/2000248980

Mariano Guerra: Papers (and other things) of the LargeSpanOfTime II

OK, the title is getting fuzzier and fuzzier, but I decided to condense some things I've been reading here.

Papers:

Bringing the Web up to Speed with WebAssembly:

I like compilers, and their implementations, so I've been following WebAssembly, this is a good place to look at.

Spanner, TrueTime & The CAP Theorem:

A blog post by google made the rounds lately with people saying that google was saying that they beat the CAP Theorem, so I went to the source. The conclusion is interesting:

Spanner reasonably claims to be an “effectively CA” system despite operating over a wide area, as it is
always consistent and achieves greater than 5 9s availability. As with Chubby, this combination is possible
in practice if you control the whole network, which is rare over the wide area. Even then, it requires
significant redundancy of network paths, architectural planning to manage correlated failures, and very
careful operations, especially for upgrades. Even then outages will occur, in which case Spanner chooses
consistency over availability.
Spanner uses two-phase commit to achieve serializability, but it uses TrueTime for external consistency,
consistent reads without locking, and consistent snapshots.

Bitcoin: A Peer-to-Peer Electronic Cash System:

Again, many people ranting and raving about bitcoin, blockchain and cryptocurrencies, what's better than go to the source, really readable paper.

CAP Twelve Years Later: How the “Rules” Have Changed:

I have a deja vu that I already read this paper, but just to be sure I read it again, interesting summary of the concepts and how they evolved over time.

LSM-trie: An LSM-tree-based Ultra-Large Key-Value Store for Small Data:

I wanted to read the LSM-tree paper and it seems I didn't look what I was clicking so instead I ended up reading the LSM-trie paper, which is really interesting and has an overview of the LSM-tree one, now I have to go and read that one too.

A prettier printer Philip Wadler:

In a previous post I mentioned that I read "The Design of a Pretty-printing Library" and I was expecting something else, well, this paper is a something else that I liked more.

Metaobject protocols: Why we want them and what else they can do:

Being an aspiring Smug Lisp Weenie I had to read this one, it's a nice paper and puts a name on some "patterns" that I've observed but couldn't describe clearly.

The Cube Data Model: A Conceptual Model and Algebra for On-Line Analytical Processing in Data Warehouses:

I've been thinking lately about the relation between Pivot Tables, Data Cubes and the things mentioned in the paper A Layered Grammar of Graphics so I started reading more about Data Cubes, I skimmed a couple papers that I forgot to register somewhere but this one was one I actually registered.

End-to-End Arguments in System Design:

Someone somewhere mentioned this paper so I went to look, it's a really good one, like the Metaobject protocol paper and other's I've read, this one is like a condensation of years of knowledge and experiences that are really interesting to read.

Books:

Object-Oriented Programming in the Beta Programming Language:

Interesting book about a really interesting (and different) object oriented programming language by the creators of Simula (aka the creators of object orientation), it explains an abstraction called "patterns" in which all other abstractions are expressed.

Project Oberon The Design of an Operating System and Compiler:

Another interesting book by Niklaus Wirth, creator of between others, Pascal, Modula and Oberon describing how to basically create computing from scratch.

I will note that I skimmed over the dense specification parts of those books since I wasn't trying to implement nor use them.

Reading:

Papers this looong week: 11 (count books as papers because why not)

Papers so far: 54

Papers in queue: don't know

Mariano Guerra: Multi-Paxos with riak_ensemble Part 1

In this post I will do the initial steps to setup a project using riak_ensemble and use its core APIs, we will do it manually in the shell on purpose, later (I hope) I will post how to build it properly in code.

First we create a new project, I'm using erlang 19.3 and rebar3 3.4.3:

rebar3 new app name=cadena

Then add riak_ensemble dependency to rebar.config, it should look like this:

{erl_opts, [debug_info]}.
{deps, [{riak_ensemble_ng, "2.4.0"}]}.

Now on 3 different terminals start 3 erlang nodes:

rebar3 shell --name node1@127.0.0.1
rebar3 shell --name node2@127.0.0.1
rebar3 shell --name node3@127.0.0.1

Run the following in every node:

Timeout = 1000.
Ensemble = root.
K1 = <<"k1">>.

application:set_env(riak_ensemble, data_root, "data/" ++ atom_to_list(node())).
application:ensure_all_started(riak_ensemble).

We are setting a variable telling riak_ensemble where to store the data for each node, node1 will store it under data/node1@127.0.0.1 node2 on data/node2@127.0.0.1 and node3 on data/node3@127.0.0.1

After that we ensure all apps that riak_ensemble requires to run are started.

You should see something like this:

ok

18:05:50.548 [info] Application lager started on node 'node1@127.0.0.1'
18:05:50.558 [info] Application riak_ensemble started on node 'node1@127.0.0.1'
{ok,[syntax_tools,compiler,goldrush,lager,riak_ensemble]}

Now on node1 run:

riak_ensemble_manager:enable().

Output:

ok

We start the riak_ensemble_manager in one node only.

Then on node2 we join node1 and node3:

riak_ensemble_manager:join('node1@127.0.0.1' ,node()).
riak_ensemble_manager:join('node3@127.0.0.1' ,node()).

Output on node2:

18:06:39.285 [info] JOIN: success
ok
remote_not_enabled

This command also generates output on node1:

18:06:24.008 [info] {root,'node1@127.0.0.1'}: Leading
18:06:39.281 [info] join(Vsn): {1,64} :: 'node2@127.0.0.1' :: ['node1@127.0.0.1']

On node3 we join node1 and node2:

riak_ensemble_manager:join('node1@127.0.0.1' ,node()).
riak_ensemble_manager:join('node2@127.0.0.1' ,node()).

Output on node 3:

18:07:36.078 [info] JOIN: success
ok

Output on node 1:

18:07:36.069 [info] join(Vsn): {1,291} :: 'node3@127.0.0.1' :: ['node1@127.0.0.1','node2@127.0.0.1']
18:07:36.074 [info] join(Vsn): {1,292} :: 'node3@127.0.0.1' :: ['node1@127.0.0.1','node2@127.0.0.1','node3@127.0.0.1']

Run this on all nodes:

riak_ensemble_manager:check_quorum(Ensemble, Timeout).
riak_ensemble_peer:stable_views(Ensemble, Timeout).
riak_ensemble_manager:cluster().

Output:

true
{ok,true}
['node1@127.0.0.1','node2@127.0.0.1','node3@127.0.0.1']

Everything seems to be ok, we have a cluster!

Now we can write something, let's set key "k1" to value "v1" on all nodes using paxos for consensus.

On node1 run:

V1 = <<"v1">>.
riak_ensemble_client:kover(node(), Ensemble, K1, V1, Timeout).

Output:

{ok,{obj,1,729,<<"k1">>,<<"v1">>}}

We can check on node2 that the value is available:

riak_ensemble_client:kget(node(), Ensemble, K1, Timeout).

Output:

{ok,{obj,1,729,<<"k1">>,<<"v1">>}}

Now we can try a different way to update a value, let's say we want to set a new value but depending on the current value or only if the current value is set to something specific, for that we use kmodify, which receives a function and calls us with the current value and sets the key to the value we return.

On node3 run:

V2 = <<"v2">>.
DefaultVal = <<"v0">>.
ModifyTimeout = 5000.

riak_ensemble_peer:kmodify(node(), Ensemble, K1,
    fun({Epoch, Seq}, CurVal) ->
        io:format("CurVal: ~p ~p ~p to ~p~n", [Epoch, Seq, CurVal, V2]),
        V2
    end,
    DefaultVal, ModifyTimeout).

Output on node 3:

{ok,{obj,1,914,<<"k1">>,<<"v2">>}}

Output on node 1:

CurVal: 1 914 <<"v1">> to <<"v2">>

The call with a function as parameter was done on node3 but it ran on node1, that's the advantage of using the Erlang virtual machine to build distributed systems.

Now let's check if the value was set on all nodes by checking it on node2:

riak_ensemble_client:kget(node(), Ensemble, K1, Timeout).

Output:

{ok,{obj,1,914,<<"k1">>,<<"v2">>}}

Now let's quit on all nodes:

q().

Let's start the cluster again to see if riak_ensemble rememers things, in 3 different terminals run:

rebar3 shell --name node1@127.0.0.1
rebar3 shell --name node2@127.0.0.1
rebar3 shell --name node3@127.0.0.1

On every node:

Timeout = 1000.
Ensemble = root.
K1 = <<"k1">>.

application:set_env(riak_ensemble, data_root, "data/" ++ atom_to_list(node())).
application:ensure_all_started(riak_ensemble).

We set the data_root again and start riak_enseble and its dependencies, after that on node1 we should see:

18:11:55.286 [info] {root,'node1@127.0.0.1'}: Leading

Now let's check that the cluster was initialized correctly:

riak_ensemble_manager:check_quorum(Ensemble, Timeout).
riak_ensemble_peer:stable_views(Ensemble, Timeout).
riak_ensemble_manager:cluster().

Output:

true
{ok,true}
['node1@127.0.0.1','node2@127.0.0.1','node3@127.0.0.1']

You can now check on any node you want if the key is still set:

riak_ensemble_client:kget(node(), Ensemble, K1, Timeout).

Output should be:

{ok,{obj,2,275,<<"k1">>,<<"v2">>}}

Check the generated files under the data folder:

$ tree data

data
├── node1@127.0.0.1
│   └── ensembles
│       ├── 1394851733385875569783788015140658786474476408261_kv
│       ├── ensemble_facts
│       └── ensemble_facts.backup
├── node2@127.0.0.1
│   └── ensembles
│       ├── ensemble_facts
│       └── ensemble_facts.backup
└── node3@127.0.0.1
    └── ensembles
            ├── ensemble_facts
            └── ensemble_facts.backup

6 directories, 7 files

To sum up, we created a project, added riak_ensemble as a dependency, started a 3 node cluster, joined all the nodes, wrote a key with a value, checked that it was available on all nodes, updated the value with a "compare and swap" operation, stopped the cluster, started it again and checked that the cluster was restarted as it was and the value was still there.

Facundo Batista: Encuentro 5.0


Tengo el agrado de anunciar, luego de mucho (demasiado) tiempo, una nueva liberación de Encuentro (que es, como ya sabrán, un programejo que permite buscar, descargar y ver contenido del Canal Encuentro, Paka Paka, BACUA, Educ.ar, Decime quien sos vos, TED y otros).

Encuentro (¡nuevo logo y tipografía!)

Esta nueva liberación se justifica principalmente por dos motivos: renovación, y cambio estético.

La renovación es porque tuvimos (junto a Diego Mascialino, que me ayudó un montón) que renovar la mitad de los backends, ya que algunos habían cambiado tanto que no andaban más y tuvimos que meterle bastante laburo para sacarlos adelante.

El cambio estético viene de la mano de Cecilia Schiebel, quien de onda (o sea, porque tuvo la idea y se ofreció ella, y además no me quiso cobrar ni un peso) renovó la estética del sitio web, y armó un nuevo ícono e imágenes para el programa.

También hay un par más de correcciones o mejoras, pero nada demasiado importante.

El detalle de la nueva versión, instaladores, etc, en la página oficial.

¡Que lo disfruten!

Juanjo Conti: Goodreads review: La pesquisa (Juan José Saer)

Lo leí durante unos días que estuve de vacaciones en París (donde transcurre la mitad de la trama) y lo disfruté mucho.

El primer capítulo (largo y sin nombre) es un relato sobre un asesino en París. Hay vestigios coloquiales pero uno no termina de saber quién está hablando.

En el segundo capítulo (igual de largo; y con esto ya completamos la mitad de las páginas del libro) muestra la escena donde la primera anécdota se está contando.

El tercer y final capítulo (más largo aún; se lleva el resto del libro) va intercalando ambas narración previas.

Hice varias marcas mientras lo leía. Si encuentro el tiempo, voy a escribir algo: Notas sobre La pesquisa.

Rating: 5/5

Original: https://www.goodreads.com/review/show/1996391293