# Installing and testing latest version of Qserv in Docker

**URL:** <https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093>\
**Category:** Data Management\
**Created:** [April 14, 2020, 10:36am UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093 "2020-04-14T10:36:59Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Teng](https://avatars.discourse-cdn.com/v4/letter/t/b782af/32.png) [@Teng](https://www.rubin.community/u/Teng)\
**Post date:** [April 14, 2020, 10:36am UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/1 "2020-04-14T10:36:59Z")

</div>

Dear Qserv developers,

Recently I’ve been trying to install and do some tests with the latest version of Qserv (by the time I’m writing this post was b3a533f).

I’m doing a simple multi-node installation on my CentOS7 localhost, using the deployment tool in the Qserv repo: admin/tools/docker/deployment/localhost/run-multinode-tests.sh, and this is my env.sh.

> VERSION=b3a533f  
> NB\_WORKERS=2
> 
> # Set nodes names
> 
> DNS\_DOMAIN=localdomain  
> MASTER=master.“$DNS\_DOMAIN”  
> for i in $(seq 1 “$NB\_WORKERS”);  
> do  
> WORKERS=“$WORKERS worker${i}.$DNS\_DOMAIN”  
> done
> 
> # Set images names
> 
> MASTER\_IMAGE=“qserv/qserv:${VERSION}\_master”  
> WORKER\_IMAGE=“qserv/qserv:${VERSION}\_worker”

The worker nodes cannot start up correctly, the error message of the worker container:

> Container timezone not modified  
> INFO: Qserv execution directory : /qserv/run  
> Starting MySQL. SUCCESS!  
> Starting xrootd./qserv/run/etc/init.d/qserv-functions: line 46: 267 Aborted (core dumped) /qserv/stack/stack/miniconda3-4.7.12-984c9f7/Linux64/xrootd/lsst-dev-g628f1b81fb/bin/xrootd -c /qserv/run/etc/lsp.cf -l @libXrdSsiLog.so -n worker -I v4 -+xrdssi /qserv/run/etc/xrdssi.cnf \>\> /qserv/run/var/log/xrootd.log 2\>&1  
> ERROR! : Manager of pid-file quit without updating file.  
> ERROR!  
> See startup logfiles : /qserv/run/var/log/xrootd.log, /qserv/run/var/log/worker/xrootd.log  
> Starting cmsd. SUCCESS!  
> Starting qserv-wmgr SUCCESS!

By looking into the container, I found the xrootd daemon would crash on start, generating a core file with no error message in the log file.

I’m wondering if this was me doing tests on a broken build or something else I’ve done wrong?  
If you need further information I can provide you.

Many Thanks,  
Teng

---

<div class="post-metadata">

**Author:** ![fritzm](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/fritzm/32/32_2.png) [@fritzm](https://www.rubin.community/u/fritzm)\
**Post date:** [April 14, 2020, 4:33pm UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/2 "2020-04-14T16:33:43Z")

</div>

Hi Teng,

The containers at the SHA you mention were chronologically the most recent built by Travis, but happen to have been built off one of Fabrice’s as-yet un-merged branches, and so aren’t guaranteed to be in working order.

Our latest SHA on master currently is 93c3b68. Could you please give it another shot with those containers and see if you achieve a better result? If not, follow up again here and we’ll be glad to help figure it out!

Thanks,  
–FritzM.

---

<div class="post-metadata">

**Author:** ![Teng](https://avatars.discourse-cdn.com/v4/letter/t/b782af/32.png) [@Teng](https://www.rubin.community/u/Teng)\
**Post date:** [April 16, 2020, 10:25am UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/3 "2020-04-16T10:25:21Z")

</div>

Hi Fritz,

Thank you for your reply, I’ll try 93c3b68.

Cheers,  
Teng

---

<div class="post-metadata">

**Author:** ![Teng](https://avatars.discourse-cdn.com/v4/letter/t/b782af/32.png) [@Teng](https://www.rubin.community/u/Teng)\
**Post date:** [April 16, 2020, 10:52am UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/4 "2020-04-16T10:52:11Z")

</div>

Hi Fritz,

It seems the error is the same with the latest version:

```auto
[qserv@worker2 bin]$ ./qserv-start.sh 
INFO: Qserv execution directory : /qserv/run
Starting MySQL. SUCCESS! 
Starting xrootd./qserv/run/etc/init.d/qserv-functions: line 46: 348 Aborted (core dumped) /qserv/stack/stack/miniconda3-4.7.12-984c9f7/Linux64/xrootd/lsst-dev-g628f1b81fb/bin/xrootd -c /qserv/run/etc/lsp.cf -l @libXrdSsiLog.so -n worker -I v4 -+xrdssi /qserv/run/etc/xrdssi.cnf >> /qserv/run/var/log/xrootd.log 2>&1
 ERROR! : Manager of pid-file quit without updating file.
 ERROR! 
See startup logfiles : /qserv/run/var/log/xrootd.log, /qserv/run/var/log/worker/xrootd.log
Starting cmsd. SUCCESS! 
Starting qserv-wmgr SUCCESS! 
[qserv@worker2 bin]$ 
[qserv@worker2 bin]$ ls
core.348 env.sh qserv-connect-mysql-proxy.sh qserv-connect-mysql-sock.sh qserv-restart.sh qserv-start.sh qserv-status.sh qserv-stop.sh worker

```

---

<div class="post-metadata">

**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\
**Post date:** [April 16, 2020, 11:03am UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/5 "2020-04-16T11:03:08Z")

</div>

That `core.348` file will be useful, if it contains data. Please try to preserve it and put it somewhere where Fritz’s team can get at it.

---

<div class="post-metadata">

**Author:** ![Teng](https://avatars.discourse-cdn.com/v4/letter/t/b782af/32.png) [@Teng](https://www.rubin.community/u/Teng)\
**Post date:** [April 16, 2020, 11:17am UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/6 "2020-04-16T11:17:38Z")

</div>

The core.348 is 82M. How could I share it with you?

Teng

---

<div class="post-metadata">

**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\
**Post date:** [April 16, 2020, 11:21am UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/7 "2020-04-16T11:21:31Z")

</div>

If it’s (only) 82MB, you could attach it to a message here using the “Upload” button in the editor.

---

<div class="post-metadata">

**Author:** ![Teng](https://avatars.discourse-cdn.com/v4/letter/t/b782af/32.png) [@Teng](https://www.rubin.community/u/Teng)\
**Post date:** [April 16, 2020, 12:10pm UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/8 "2020-04-16T12:10:37Z")

</div>

There’s a 4Mb limit on the attachment. I can email it to anybody who wants it.

---

<div class="post-metadata">

**Author:** ![fritzm](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/fritzm/32/32_2.png) [@fritzm](https://www.rubin.community/u/fritzm)\
**Post date:** [April 16, 2020, 6:30pm UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/9 "2020-04-16T18:30:58Z")

</div>

Hi Teng, thanks for checking master, that will make it easier for us to troubleshoot.

We’d be interested to see:

- core file
- the two called-out log files
- /qserv/run/etc/xrdssi.cnf from inside the container

If you’d like to give email a shot for the core file, you could hit me at fritzm@slac.stanford.edu. Otherwise, if you put it up somewhere on AFS we’ll come and get it?

---

<div class="post-metadata">

**Author:** ![fritzm](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/fritzm/32/32_2.png) [@fritzm](https://www.rubin.community/u/fritzm)\
**Post date:** [April 16, 2020, 6:32pm UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/10 "2020-04-16T18:32:17Z")

</div>

Oh, also:

- /qserv/run/etc/lsp.cf from inside the container

---

<div class="post-metadata">

**Author:** ![Teng](https://avatars.discourse-cdn.com/v4/letter/t/b782af/32.png) [@Teng](https://www.rubin.community/u/Teng)\
**Post date:** [April 17, 2020, 10:36am UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/11 "2020-04-17T10:36:09Z")

</div>

> [@fritzm](#):
>
> [fritzm@slac.stanford.edu](mailto:fritzm@slac.stanford.edu)

Hi Fritz,

I’ve sent you the files, the /qserv/run/var/log/worker/xrootd.log doesn’t exist so there’s only one xrootd.log.

Thank,  
Teng

---

<div class="post-metadata">

**Author:** ![fritzm](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/fritzm/32/32_2.png) [@fritzm](https://www.rubin.community/u/fritzm)\
**Post date:** [April 20, 2020, 4:20pm UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/12 "2020-04-20T16:20:59Z")

</div>

Hi Teng – your email has apparently not made it through. Checked my spam filter too, but have come up empty.

Any other way than email you might get us that core file? public\_html somewhere? Google drive? AFS? …?

cheers,  
–FritzM.

---

<div class="post-metadata">

**Author:** ![Teng](https://avatars.discourse-cdn.com/v4/letter/t/b782af/32.png) [@Teng](https://www.rubin.community/u/Teng)\
**Post date:** [April 21, 2020, 1:42pm UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/13 "2020-04-21T13:42:38Z")

</div>

> [@Teng](#):
>
> 93c3b68

Sorry for the delay. I’ve made a google drive share:

> **[debug\_qserv\_93c3b68.tar.gz](https://drive.google.com/file/d/1jZ9m_FM7afRJ2wgClviAi0acuwSLbnDk/view?usp=sharing)**
>
> Google Drive file.

---

<div class="post-metadata">

**Author:** ![fritzm](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/fritzm/32/32_2.png) [@fritzm](https://www.rubin.community/u/fritzm)\
**Post date:** [April 22, 2020, 8:02pm UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/14 "2020-04-22T20:02:04Z")

</div>

Thanks, Teng – got it, taking a look now…

---

<div class="post-metadata">

**Author:** ![fritzm](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/fritzm/32/32_2.png) [@fritzm](https://www.rubin.community/u/fritzm)\
**Post date:** [April 22, 2020, 8:39pm UTC](https://www.rubin.community/t/installing-and-testing-latest-version-of-qserv-in-docker/4093/15 "2020-04-22T20:39:30Z")

</div>

Hi Teng,

I managed to get a backtrace from that core file, and what we find is a `boost::uuids::entropy_error` being thrown from `boost::uuids::random_generator()`.

This is likely a result of the Qserv worker Docker container being hosted on an older kernel that is missing some recent syscalls. Could you let us know what the kernel version is that is available on your docker host?
