make dev failing on develop for redis can not get port 6739

[kushaldas]

Description

make dev is failing only on my local laptop with error related to Redis connection.

Steps to Reproduce

• make dev

Expected Behavior

• A dev container will start.

Actual Behavior

Successfully tagged securedrop-test-xenial-py3:latest
************************************************************
Exposed Docker ports will be available on localhost with
Port offset: 0
Source interface: 8080
Journalist interface: 8081
VNC: 5909
************************************************************
9:C 06 Oct 10:13:26.547 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.0.6 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 9
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

9:M 06 Oct 10:13:26.549 # Server started, Redis version 3.0.6
9:M 06 Oct 10:13:26.549 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
9:M 06 Oct 10:13:26.549 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
9:M 06 Oct 10:13:26.549 # Can't handle RDB format version 9
9:M 06 Oct 10:13:26.549 # Fatal error loading the DB: Invalid argument. Exiting.
███ Generating securedrop/config.py...
>>> Sass is watching for changes. Press Ctrl-C to stop.

[Listen warning]:
  Listen will be polling for changes. Learn more at https://github.com/guard/listen#polling-fallback.
seq  name             file                                                      
---  ---------------  ----------------------------------------------------------
0    main             /var/lib/securedrop/db.sqlite                             
gpg: key 28271441: public key "SecureDrop Test/Development (DO NOT USE IN PRODUCTION)" imported
gpg: Total number processed: 1
gpg:               imported: 1
Test user successfully added: username=journalist, password=correct horse battery staple profanity oil chewy, otp_secret=JHCOGO7VCER3EJ4L, is_admin=True
Test user successfully added: username=dellsberg, password=correct horse battery staple profanity oil chewy, otp_secret=JHCOGO7VCER3EJ4L, is_admin=False
Test user successfully added: username=clarkkent, password=correct horse battery staple profanity oil chewy, otp_secret=JHCOGO7VCER3EJ4L, is_admin=False
Traceback (most recent call last):
  File "/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/redis/connection.py", line 526, in connect
    sock = self._connect()
  File "/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/redis/connection.py", line 583, in _connect
    raise err
  File "/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/redis/connection.py", line 571, in _connect
    sock.connect(socket_address)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./create-dev-data.py", line 162, in <module>
    main(args.staging)
  File "./create-dev-data.py", line 64, in main
    i, num_sources, journalist_who_replied=journalist_tobe_deleted
  File "./create-dev-data.py", line 131, in create_source_and_submissions
    [current_app.crypto_util.get_fingerprint(source.filesystem_id),
  File "/home/kdas/code/focal/securedrop/securedrop/crypto_util.py", line 283, in get_fingerprint
    fingerprint = self.redis.hget(self.REDIS_FINGERPRINT_HASH, name)
  File "/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/redis/client.py", line 2713, in hget
    return self.execute_command('HGET', name, key)
  File "/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/redis/client.py", line 836, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
  File "/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/redis/connection.py", line 1059, in get_connection
    connection.connect()
  File "/opt/venvs/securedrop-app-code/lib/python3.5/site-packages/redis/connection.py", line 531, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 99 connecting to localhost:6379. Cannot assign requested address.
make: *** [Makefile:176: dev] Error 1

Comments

Suggestions to fix, any other relevant information.

[emkll]

Thanks for opening @kushaldas I just ran into the same issue myself, deleted dump.rdb in the securedrop folder and it resolved for me. However, I have no idea how dump.rdb appeared in the first place.

[eloquence]

Unable to reproduce. I also don't get the fancy Redis ASCII Art in the output.

[kushaldas]

Thanks for opening @kushaldas I just ran into the same issue myself, deleted dump.rdb in the securedrop folder and it resolved for me. However, I have no idea how dump.rdb appeared in the first place.

I found the same file and removing it solved the error.

[kushaldas]

Digging into the dump.rdb file and it says:

REDIS0009<FA>   redis-ver^E5.0.7<FA>

The 5.0.7 is the version available in Focal, in Xenial has 3.0.6 version.

[rmol]

This file is getting generated by Redis in the container any time it runs long enough and/or sees enough activity (check /etc/redis/redis.conf in the container, under the save 900 1, save 300 10, etc. lines). It gets created on develop, under Xenial, too.

The only problem is that the format has jumped three versions between Xenial and Focal, so switching between the releases, running the test suite, or keeping the dev server up long enough under Focal results in a file that's unusable by Xenial.

A simple fix is probably removing the file when the container boots, maybe in the run_redis function in securedrop/bin/dev-deps. Only these containers will be creating that file in that location, and no good can come of the Redis database persisting across container runs.