Ask Your Question
0

Glance in combination with docker provokes "broken pipe" error when uploading a bigger docker image file

asked 2015-07-22 05:02:02 -0500

holger-king gravatar image

updated 2015-07-23 09:56:58 -0500

Dear OpenStack community,

when trying to upload a bigger docker image (here: "supertaihei02/docker-centos-lamp:lamp", 883.8 MB) to glance via:

[root@rb-openstack-d ~(keystone_admin)]# docker pull "supertaihei02/docker-centos-lamp:lamp"
[root@rb-openstack-d ~(keystone_admin)]# docker save "supertaihei02/docker-centos-lamp:lamp" | glance image-create --is-public=True --container-format=docker --disk-format=raw --name "supertaihei02/docker-centos-lamp:lamp"
Error finding address for http://10.116.64.10:9292/v1/images: [Errno 32] Broken pipe
time="2015-07-22T11:27:42+02:00" level=fatal msg="write /dev/stdout: broken pipe"

we get the above "broken pipe" error.

When uploading a "rastasheep/ubuntu-sshd:14.04" (251.4 MB) to glance using the same approach and commands it works fine.

Is there an upload threshold that prevents uploading images greater than a certain size?

edit retag flag offensive close merge delete

Comments

Size is about 883 MB, I have experience with uploading > 1.2 GB docker images. Size is not a reason

dbaxps gravatar imagedbaxps ( 2015-07-22 08:46:34 -0500 )edit

We can reproduce it - regularly. And we see the following DOCKER error in "/var/log/messages":

openstack-d docker: time="2015-07-23T16:20:10+02:00" level=error msg="Handler for GET /images/{name:.*}/get returned error: write unix @: broken pipe"

That means it's related to docker!

holger-king gravatar imageholger-king ( 2015-07-23 09:25:41 -0500 )edit

When first saving the docker image to a tar via:

docker save -o <tar_file_name> <image_id>

and finally importing it to glance via:

glance image-create --is-public=True   --container-format=docker --disk-format=raw --name supertaihei02/docker-centos-lamp:lamp < <tar_file_name>

it works!

holger-king gravatar imageholger-king ( 2015-07-23 09:40:16 -0500 )edit

Which docker version do you use? We use:

docker-1.6.2-14.el7.x86_64
holger-king gravatar imageholger-king ( 2015-07-23 09:42:03 -0500 )edit

docker-selinux-1.6.2-14.el7.centos.x86_64
docker-1.6.2-14.el7.centos.x86_64

dbaxps gravatar imagedbaxps ( 2015-07-23 10:23:04 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
0

answered 2015-07-22 08:20:55 -0500

dbaxps gravatar image

updated 2015-07-22 11:33:03 -0500

Cannot reproduce :-

[root@ServerCentOS71 ~(keystone_admin)]# docker pull "supertaihei02/docker-centos-lamp:lamp"
Trying to pull repository docker.io/supertaihei02/docker-centos-lamp ...
fd8dbcf7c7a8: Download complete 
511136ea3c5a: Download complete 
34e94e67e63a: Download complete 
0c752394b855: Download complete 
313bc652d65e: Download complete 
7bca8e6c58d0: Download complete 
02c7dd4b7b17: Download complete 
1ccb205c0045: Download complete 
67c95f15098c: Download complete 
265c1884886d: Download complete 
397ee04ea830: Download complete 
d57e43613b5c: Download complete 
2614d5f8ae73: Download complete 
bcd82bfaa75d: Download complete 
2513dc5a7634: Download complete 
99c99df8dd45: Download complete 
2c7af5e056c6: Download complete 
474020a06892: Download complete 
72ca693b56ec: Download complete 
32538e0f2a30: Download complete 
67d8a91c730d: Download complete 
a3f485a396f4: Download complete 
ad78608b9dec: Download complete 
08fbdbd00577: Download complete 
1fa3dcbd7265: Download complete 
789146ed1568: Download complete 
297a56c9f440: Download complete 
1b1962c27933: Download complete 
de86771fcddf: Download complete 
2fc57b1bb564: Download complete 
b9979c770dbd: Download complete 
dfa087672ba1: Download complete 
108d43aa66f4: Download complete 
4573e6861576: Download complete 
Status: Downloaded newer image for docker.io/supertaihei02/docker-centos-lamp:lamp
[root@ServerCentOS71 ~(keystone_admin)]# docker save supertaihei02/docker-centos-lamp:lamp | glance image-create --is-public=True   --container-format=docker --disk-format=raw --name supertaihei02/docker-centos-lamp:lamp
+------------------+---------------------------------------+
| Property         | Value                                 |
+------------------+---------------------------------------+
| checksum         | 24169bb5bf7b8e218af565b9105d231f      |
| container_format | docker                                |
| created_at       | 2015-07-22T13:15:49.000000            |
| deleted          | False                                 |
| deleted_at       | None                                  |
| disk_format      | raw                                   |
| id               | b317fed0-4618-406b-925f-954eb2aa35f7  |
| is_public        | True                                  |
| min_disk         | 0                                     |
| min_ram          | 0                                     |
| name             | supertaihei02/docker-centos-lamp:lamp |
| owner            | 616a4800e9a541b78e4aba836314883d      |
| protected        | False                                 |
| size             | 899017728                             |
| status           | active                                |
| updated_at       | 2015-07-22T13:17:13.000000            |
| virtual_size     | None                                  |
+------------------+---------------------------------------+

Next

[root@ServerCentOS71 ~(keystone_admin)]# docker images |grep lamp
docker.io/supertaihei02/docker-centos-lamp   lamp                fd8dbcf7c7a8        13 months ago       883.8 MB

Nova-Doker container started

[root@ServerCentOS71 ~(keystone_admin)]# docker ps

CONTAINER ID        IMAGE                                   COMMAND               CREATED             STATUS              PORTS               NAMES
6a1e0a782a7d        supertaihei02/docker-centos-lamp:lamp   "/usr/bin/monit -I"   4 minutes ago       Up 4 minutes                            nova-8ade872c-45dc-40b0-be38-a5ab61d7de0f   
[root@ServerCentOS71 ~(keystone_admin)]# docker logs 6a1e0a782a7d
monit: generated unique Monit id 78e9d47e7325f30d9ba76b908278921b and stored to '/var/monit/id'

image description

  [root@ServerCentOS71 ~(keystone_demo)]# nova list | grep LAMP
    | 8ade872c-45dc-40b0-be38-a5ab61d7de0f | CentOSLAMP       | ACTIVE  | -          | Running     | private_demo=30.0.0.24, 192.168.1.165 |
    [root@ServerCentOS71 ~(keystone_demo)]# docker ps
    CONTAINER ID        IMAGE                                   COMMAND               CREATED             STATUS              PORTS               NAMES
    6a1e0a782a7d        supertaihei02/docker-centos-lamp:lamp   "/usr/bin/monit -I"   3 hours ago         Up About a minute                       nova-8ade872c-45dc-40b0-be38-a5ab61d7de0f

Login to domain as instructed

[root@ServerCentOS71 ~(keystone_demo)]# ssh -p 22 -l guest 192.168.1.165
guest@192.168.1.165's password: 
Last login: Thu Jul 23 01:20:31 2015 from 192.168.1.87
[guest@instance-00000010 ~]$ id
uid=500(guest) gid=500(guest) groups=500(guest)
[guest@instance-00000010 ~]$ sudo su -
[root@instance-00000010 ~]# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 01:22 ?        00:00:00 /usr/bin/monit -I
root         7     1  0 01:22 ?        00:00:00 [sshd] <defunct>
root        18     1  0 01:22 ?        00:00:00 /usr/sbin/sshd
root        21     1  0 01:22 ?        00:00:00 [mysqld] <defunct>
root        55     1  0 01:22 ?        00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --s
mysql      264    55  0 01:22 ?        00:00:00 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql
root       295     1  0 01:22 ?        00:00:00 [httpd] <defunct>
root       300     1  0 01:22 ?        00:00:00 /usr/sbin/httpd
apache     302   300  0 01:22 ?        00:00:00 /usr/sbin/httpd
apache     303   300 ...
(more)
edit flag offensive delete link more
0

answered 2015-07-24 03:06:40 -0500

holger-king gravatar image

The problem is related to docker (here: docker-1.6.2-14.el7.x86_64) as the log file "/var/log/messages" shows:

When first saving the docker image to a tar file via:

docker save -o <tar_file_name> <image_id>

and finally importing it to glance via:

glance image-create --is-public=True   --container-format=docker --disk-format=raw --name supertaihei02/docker-centos-lamp:lamp < <tar_file_name>

it works successfully - without provoking the broken pipe error!

Finally, the pipe-based combination of docker and glance seems to be not robust enough. Especially when taking a look in the "/var/log/messages" docker complains about:

docker: time="2015-07-23T16:20:10+02:00" level=info msg="-job image_tarlayer(511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158) = OK (0)"
docker: time="2015-07-23T16:20:10+02:00" level=info msg="+job image_get(511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158)"
docker: time="2015-07-23T16:20:10+02:00" level=info msg="-job image_get(511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158) = OK (0)"
docker: write unix @: broken pipe
docker: time="2015-07-23T16:20:10+02:00" level=info msg="-job image_export(supertaihei02/docker-centos-lamp:lamp) = ERR (1)"
docker: time="2015-07-23T16:20:10+02:00" level=error msg="Handler for GET /images/{name:.*}/get returned error: write unix @: broken pipe"
docker: time="2015-07-23T16:20:10+02:00" level=error msg="HTTP Error: statusCode=500 write unix @: broken pipe"
docker: 2015/07/23 16:20:10 http: multiple response.WriteHeader calls
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2015-07-22 05:02:02 -0500

Seen: 1,736 times

Last updated: Jul 24 '15