init script returns success even if riak does not start

[jsmartin]

This is on riak 1.3.1, Ubuntu 12.04 package.

    root@riak-1:~# service riak start
    Attempting to restart script through sudo -H -u riak
    Riak failed to start within 15 seconds,
    see the output of 'riak console' for more information.
    If you want to wait longer, set the environment variable
    WAIT_FOR_ERLANG to the number of seconds to wait.
    root@riak-1:~# echo $?
    0

$? should return 0 if the service has failed to start.

[jaredmorrow]

Riak does not use node_package 'yet'. The plan is to convert it for 1.4, this bug will be fixed then.

[jaredmorrow]

@jsmartin so looking into it, strangely enough debian/ubuntu do not return error exit codes on init scripts doing start() and stop(). I made the ssh script fail to start and it indeed also returned 0.

Pages like this http://www.thegeekstuff.com/2012/03/lsbinit-script/ and several others tell you to use the log_end_msg function which we use in node_package.

This LSB page seems to suggest that they should return failures. But the /lib/lsb/init-functions on Ubuntu/Debian log_end_msg does not actually return anything besides 0, but relies on logging instead.

I'm perplexed to say the least, as I'd expect log_end_msg 1 to also exit with a 1.

I will investigate further on what the "right" way to do it is.

[jaredmorrow]

Looks like I'm not the only who's confused, seems like the debian policy of exiting with a 0 exit code is clashing with the LSB.

[jaredmorrow]

So turns out this is not a bug according to debian policy and will close this for now.

[hectcastro]

Was about to create a new issue for this, until @jaredmorrow pointed me at this one.

---

While working on the vagrant-riak-cs-cluster project, I noticed that the Ubuntu init script is failing silently. For what it's worth, the CentOS init script seems to be detecting failures correctly.

Ubuntu:

vagrant@riak1:~$ uname -a
Linux riak1 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
vagrant@riak1:~$ sudo /etc/init.d/riak start
Attempting to restart script through sudo -H -u riak
Riak failed to start within 15 seconds,
see the output of 'riak console' for more information.
If you want to wait longer, set the environment variable
WAIT_FOR_ERLANG to the number of seconds to wait.
vagrant@riak1:~$ echo $?
0
vagrant@riak1:~$ sudo riak ping
Attempting to restart script through sudo -H -u riak
!!!!
!!!! WARNING: ulimit -n is 1024; 4096 is the recommended minimum.
!!!!
Node 'riak@33.33.33.10' not responding to pings.

CentOS:

[vagrant@riak1 ~]$ uname -a
Linux riak1 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
[vagrant@riak1 ~]$ sudo /etc/init.d/riak start
Starting Riak: Riak failed to start within 15 seconds,
see the output of 'riak console' for more information.
If you want to wait longer, set the environment variable
WAIT_FOR_ERLANG to the number of seconds to wait.
                                                           [FAILED]
[vagrant@riak1 ~]$ echo $?
1
[vagrant@riak1 ~]$ sudo riak ping
Attempting to restart script through sudo -H -u riak
Node 'riak@33.33.33.10' not responding to pings.

In this particular example, Riak failed to start because the Riak CS backend was being referenced before the Riak CS package was installed:

vagrant@riak1:~$ sudo riak console
Attempting to restart script through sudo -H -u riak
!!!!
!!!! WARNING: ulimit -n is 1024; 4096 is the recommended minimum.
!!!!
Exec: /usr/lib/riak/erts-5.9.1/bin/erlexec -boot /usr/lib/riak/releases/1.3.1/riak             -embedded -config /etc/riak/app.config             -pa /usr/lib/riak/lib/basho-patches             -args_file /etc/riak/vm.args -- console
Root: /usr/lib/riak
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:1:1] [async-threads:64] [kernel-poll:true]

/usr/lib/riak/lib/os_mon-2.2.9/priv/bin/memsup: Erlang has closed.
                                                                   Erlang has closed
                                                                                    {"Kernel pid terminated",application_controller,"{application_start_failure,riak_kv,{invalid_storage_backend,{riak_kv_app,start,[normal,[]]}}}"}

Crash dump was written to: /var/log/riak/erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,riak_kv,{invalid_storage_backend,{riak_kv_app,start,[normal,[]]}}})

[jaredmorrow]

After many people complaining about this, I will change the implementation to meet this: http://refspecs.linuxbase.org/LSB_3.1.1/LSB-Core-generic/LSB-Core-generic/iniscrptact.html

[hectcastro]

Nice.