Discussion:
[HEADS UP] Ceph-14.x.x, dropping 32-bit archs
Kaleb S. KEITHLEY
2018-12-05 13:14:30 UTC
Permalink
Ceph 14.x.x (Nautilus) will no longer be built on i686 and armv7hl archs
starting in fedora-30/rawhide.

The upstream project doesn't support it. The armv7hl builders don't have
enough memory (or address space) to build some components.

And the other active maintainer (branto) and I don't have cycles to
devote to keeping it building on 32-bit archs.

(FWIW, currently ceph-12.2.9 (luminous) is in rawhide, f29, and f28 and
it has packages for i686 and armv7hl for people who want to run ceph on
32-bit archs.)

--

Kaleb
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/de
Marcin Juszkiewicz
2018-12-05 13:23:49 UTC
Permalink
Post by Kaleb S. KEITHLEY
Ceph 14.x.x (Nautilus) will no longer be built on i686 and armv7hl archs
starting in fedora-30/rawhide.
The upstream project doesn't support it. The armv7hl builders don't have
enough memory (or address space) to build some components.
BTW - how much memory is needed to build Ceph 14?
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fe
Dan Horák
2018-12-05 13:34:46 UTC
Permalink
On Wed, 5 Dec 2018 14:23:49 +0100
Post by Marcin Juszkiewicz
Post by Kaleb S. KEITHLEY
Ceph 14.x.x (Nautilus) will no longer be built on i686 and armv7hl
archs starting in fedora-30/rawhide.
The upstream project doesn't support it. The armv7hl builders don't
have enough memory (or address space) to build some components.
BTW - how much memory is needed to build Ceph 14?
have you tried building with reduced debuginfo, eg. -g1 or even -g0?

I wonder how much broken deps it will cause.


Dan
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/deve
Kaleb S. KEITHLEY
2018-12-05 13:45:19 UTC
Permalink
Post by Dan Horák
On Wed, 5 Dec 2018 14:23:49 +0100
Post by Marcin Juszkiewicz
Post by Kaleb S. KEITHLEY
Ceph 14.x.x (Nautilus) will no longer be built on i686 and armv7hl
archs starting in fedora-30/rawhide.
The upstream project doesn't support it. The armv7hl builders don't
have enough memory (or address space) to build some components.
BTW - how much memory is needed to build Ceph 14?
More — apparently — than the armv7hl builders have. :-) branto may know.
Post by Dan Horák
have you tried building with reduced debuginfo, eg. -g1 or even -g0?
branto told me that he has tried all the different optimization levels.
Post by Dan Horák
I wonder how much broken deps it will cause.
Don't know. Hence this heads up warning.

--

Kaleb
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorapro
Marcin Juszkiewicz
2018-12-05 14:02:41 UTC
Permalink
Post by Kaleb S. KEITHLEY
Post by Dan Horák
On Wed, 5 Dec 2018 14:23:49 +0100
Post by Marcin Juszkiewicz
Post by Kaleb S. KEITHLEY
Ceph 14.x.x (Nautilus) will no longer be built on i686 and armv7hl
archs starting in fedora-30/rawhide.
The upstream project doesn't support it. The armv7hl builders don't
have enough memory (or address space) to build some components.
BTW - how much memory is needed to build Ceph 14?
More — apparently — than the armv7hl builders have. :-) branto may know.
Random Fedora armhf builder hardware info:

-------------------------------------------------------------------------------------------------------
CPU info:
Architecture: armv7l
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Model: 1
Model name: ARMv7 Processor rev 1 (v7l)
BogoMIPS: 100.00
Flags: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm


Memory:
total used free shared buff/cache available
Mem: 24929616 103092 24375448 348 451076 24524500
Swap: 18869244 21348 18847896


Storage:
Filesystem Size Used Avail Use% Mounted on
/dev/vda2 135G 5.7G 122G 5% /
-------------------------------------------------------------------------------------------------------

Seriously 24 GB of ram + 18 GB of swap is not enough to build ceph? That's
more real memory than x86-64 builder have (15 387 432 ram + 134 216 700 swap).

I understand "we drop because upstream does not care about 32bit" reason.
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraprojec
Dan Horák
2018-12-05 14:08:22 UTC
Permalink
On Wed, 5 Dec 2018 15:02:41 +0100
Post by Marcin Juszkiewicz
Post by Kaleb S. KEITHLEY
Post by Dan Horák
On Wed, 5 Dec 2018 14:23:49 +0100
Post by Marcin Juszkiewicz
Post by Kaleb S. KEITHLEY
Ceph 14.x.x (Nautilus) will no longer be built on i686 and
armv7hl archs starting in fedora-30/rawhide.
The upstream project doesn't support it. The armv7hl builders
don't have enough memory (or address space) to build some
components.
BTW - how much memory is needed to build Ceph 14?
More — apparently — than the armv7hl builders have. :-) branto may know.
-------------------------------------------------------------------------------------------------------
Architecture: armv7l
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Model: 1
Model name: ARMv7 Processor rev 1 (v7l)
BogoMIPS: 100.00
Flags: half thumb fastmult vfp edsp neon vfpv3 tls
vfpv4 idiva idivt vfpd32 lpae evtstrm
total used free shared buff/cache
available Mem: 24929616 103092 24375448
348 451076 24524500 Swap: 18869244 21348
18847896
Filesystem Size Used Avail Use% Mounted on
/dev/vda2 135G 5.7G 122G 5% /
-------------------------------------------------------------------------------------------------------
Seriously 24 GB of ram + 18 GB of swap is not enough to build ceph?
That's more real memory than x86-64 builder have (15 387 432 ram +
134 216 700 swap).
I understand "we drop because upstream does not care about 32bit"
reason. _______________________________________________
The problem is usually in the 4GB address space on 32-bit platforms
(much less is available for user data), when it builds large C++
codebase. Either the compiler OOMs or the linker OOMs. That's why
reducing the generated debuginfos often helps.


Dan
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/
Daniel P. Berrangé
2018-12-05 14:31:10 UTC
Permalink
Post by Kaleb S. KEITHLEY
Post by Dan Horák
On Wed, 5 Dec 2018 14:23:49 +0100
Post by Marcin Juszkiewicz
Post by Kaleb S. KEITHLEY
Ceph 14.x.x (Nautilus) will no longer be built on i686 and armv7hl
archs starting in fedora-30/rawhide.
The upstream project doesn't support it. The armv7hl builders don't
have enough memory (or address space) to build some components.
Is there any consideration given to only building the ceph client
pieces on 32-bit ? Presumably those parts are simpler and thus
not likely to hit the address/memory limits, and be more tractable
for supporting ?

I very much doubt people would run ceph server parts on 32-bit,
so any usage of ceph on 32-bit is likely to be limited to the
client pieces
Post by Kaleb S. KEITHLEY
Post by Dan Horák
Post by Marcin Juszkiewicz
BTW - how much memory is needed to build Ceph 14?
More — apparently — than the armv7hl builders have. :-) branto may know.
Post by Dan Horák
have you tried building with reduced debuginfo, eg. -g1 or even -g0?
branto told me that he has tried all the different optimization levels.
Post by Dan Horák
I wonder how much broken deps it will cause.
Don't know. Hence this heads up warning.
repoquery can report on direct dependancies that would be broken by
any ceph packages being removed. There could be transitive ripples
out from there. From my own POV this would impact qemu & libvirt,
which would need to conditionally turn off their rbd support for
those archs.

Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archi
Richard W.M. Jones
2018-12-05 16:36:30 UTC
Permalink
Post by Daniel P. Berrangé
Post by Dan Horák
On Wed, 5 Dec 2018 14:23:49 +0100
Post by Kaleb S. KEITHLEY
Ceph 14.x.x (Nautilus) will no longer be built on i686 and armv7hl
archs starting in fedora-30/rawhide.
The upstream project doesn't support it. The armv7hl builders don't
have enough memory (or address space) to build some components.
Is there any consideration given to only building the ceph client
pieces on 32-bit ? Presumably those parts are simpler and thus
not likely to hit the address/memory limits, and be more tractable
for supporting ?
I very much doubt people would run ceph server parts on 32-bit,
so any usage of ceph on 32-bit is likely to be limited to the
client pieces
As Dan says, I'd like to know if you (Kaleb) considered building only
the client bits (librbd1 I think?). It's something which libguestfs
needs too albeit indirectly.

Assuming I've got the right command, the complete list of reverse
dependencies for the client side of Ceph is:

# repoquery -q --whatrequires 'librbd.so.1()(64bit)'
ceph-common-1:12.2.8-1.fc29.x86_64
ceph-common-1:12.2.9-1.fc29.x86_64
ceph-test-1:12.2.8-1.fc29.x86_64
ceph-test-1:12.2.9-1.fc29.x86_64
fio-0:3.7-2.fc29.x86_64
librbd-devel-1:12.2.8-1.fc29.x86_64
librbd-devel-1:12.2.9-1.fc29.x86_64
libvirt-daemon-driver-storage-rbd-0:4.7.0-1.fc29.x86_64
python-rbd-1:12.2.8-1.fc29.x86_64
python-rbd-1:12.2.9-1.fc29.x86_64
python3-rbd-1:12.2.8-1.fc29.x86_64
python3-rbd-1:12.2.9-1.fc29.x86_64
qemu-block-rbd-2:3.0.0-1.fc29.x86_64
qemu-block-rbd-2:3.0.0-2.fc29.x86_64
rbd-fuse-1:12.2.8-1.fc29.x86_64
rbd-fuse-1:12.2.9-1.fc29.x86_64
rbd-nbd-1:12.2.8-1.fc29.x86_64
rbd-nbd-1:12.2.9-1.fc29.x86_64
scsi-target-utils-rbd-0:1.0.70-4.fc28.x86_64

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fedoraproject.org
Richard W.M. Jones
2018-12-05 16:39:33 UTC
Permalink
Forgot about librados, so there's actually a much bigger list:

# dnf repoquery -q --whatrequires 'librados.so.2()(64bit)'
ceph-common-1:12.2.8-1.fc29.x86_64
ceph-common-1:12.2.9-1.fc29.x86_64
ceph-radosgw-1:12.2.8-1.fc29.x86_64
ceph-radosgw-1:12.2.9-1.fc29.x86_64
ceph-test-1:12.2.8-1.fc29.x86_64
ceph-test-1:12.2.9-1.fc29.x86_64
fio-0:3.7-2.fc29.x86_64
librados-devel-1:12.2.8-1.fc29.x86_64
librados-devel-1:12.2.9-1.fc29.x86_64
libradosstriper1-1:12.2.8-1.fc29.x86_64
libradosstriper1-1:12.2.9-1.fc29.x86_64
librbd1-1:12.2.8-1.fc29.x86_64
librbd1-1:12.2.9-1.fc29.x86_64
librgw2-1:12.2.8-1.fc29.x86_64
librgw2-1:12.2.9-1.fc29.x86_64
libvirt-daemon-driver-storage-rbd-0:4.7.0-1.fc29.x86_64
nfs-ganesha-0:2.7.0-3.fc29.x86_64
nfs-ganesha-0:2.7.1-2.fc29.x86_64
nfs-ganesha-rados-grace-0:2.7.0-3.fc29.x86_64
nfs-ganesha-rados-grace-0:2.7.1-2.fc29.x86_64
python-rados-1:12.2.8-1.fc29.x86_64
python-rados-1:12.2.9-1.fc29.x86_64
python-rbd-1:12.2.8-1.fc29.x86_64
python-rbd-1:12.2.9-1.fc29.x86_64
python-rgw-1:12.2.8-1.fc29.x86_64
python-rgw-1:12.2.9-1.fc29.x86_64
python2-cradox-0:2.1.0-2.fc29.x86_64
python3-cradox-0:2.1.0-2.fc29.x86_64
python3-rados-1:12.2.8-1.fc29.x86_64
python3-rados-1:12.2.9-1.fc29.x86_64
python3-rbd-1:12.2.8-1.fc29.x86_64
python3-rbd-1:12.2.9-1.fc29.x86_64
python3-rgw-1:12.2.8-1.fc29.x86_64
python3-rgw-1:12.2.9-1.fc29.x86_64
qemu-block-rbd-2:3.0.0-1.fc29.x86_64
qemu-block-rbd-2:3.0.0-2.fc29.x86_64
rbd-fuse-1:12.2.8-1.fc29.x86_64
rbd-fuse-1:12.2.9-1.fc29.x86_64
rbd-mirror-1:12.2.8-1.fc29.x86_64
rbd-mirror-1:12.2.9-1.fc29.x86_64
rbd-nbd-1:12.2.8-1.fc29.x86_64
rbd-nbd-1:12.2.9-1.fc29.x86_64
scsi-target-utils-rbd-0:1.0.70-4.fc28.x86_64
xrootd-ceph-1:4.8.4-2.fc29.x86_64
xrootd-ceph-1:4.8.5-2.fc29.x86_64

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@lists.fe
Tomasz Torcz
2018-12-05 17:02:11 UTC
Permalink
“Client bits” also means stuff needed for accessing file-system
interface of Ceph:
– nfs-ganesha-ceph.x86_64
– ceph-fuse

and for some users – the rados gateway bits:
– nfs-ganesha-rgw.x86_64
- ceph-radosgw.x86_64

--
Tomasz Torcz 72->| 80->|
xmpp: ***@chrome.pl 72->| 80->|
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/***@li
Peter Robinson
2018-12-06 02:50:42 UTC
Permalink
Kaleb,

Firstly the title is misleading as there was no heads up, a heads up
is notice before you actually push the change, not when you do the
change.
Post by Kaleb S. KEITHLEY
Ceph 14.x.x (Nautilus) will no longer be built on i686 and armv7hl archs
starting in fedora-30/rawhide.
The upstream project doesn't support it. The armv7hl builders don't have
enough memory (or address space) to build some components.
And the other active maintainer (branto) and I don't have cycles to
devote to keeping it building on 32-bit archs.
(FWIW, currently ceph-12.2.9 (luminous) is in rawhide, f29, and f28 and
it has packages for i686 and armv7hl for people who want to run ceph on
32-bit archs.)
As others have asked in the thread can we possibly build client only?
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/
Kaleb S. KEITHLEY
2018-12-06 12:04:12 UTC
Permalink
Post by Peter Robinson
Kaleb,
Firstly the title is misleading as there was no heads up, a heads up
is notice before you actually push the change, not when you do the
change.
I suggest you take this up with branto. He's the one who built it
without 32-bit archs without any warning. I only found out about it when
I got build notices from koji (or pagure or whatever.)

You would have preferred no notice at all?
Post by Peter Robinson
Post by Kaleb S. KEITHLEY
Ceph 14.x.x (Nautilus) will no longer be built on i686 and armv7hl archs
starting in fedora-30/rawhide.
The upstream project doesn't support it. The armv7hl builders don't have
enough memory (or address space) to build some components.
And the other active maintainer (branto) and I don't have cycles to
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Post by Peter Robinson
Post by Kaleb S. KEITHLEY
devote to keeping it building on 32-bit archs.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Post by Peter Robinson
Post by Kaleb S. KEITHLEY
(FWIW, currently ceph-12.2.9 (luminous) is in rawhide, f29, and f28 and
it has packages for i686 and armv7hl for people who want to run ceph on
32-bit archs.)
As others have asked in the thread can we possibly build client only?
We?
Post by Peter Robinson
And the other active maintainer (branto) and I don't have cycles to
devote to keeping (any part of) it building on 32-bit archs.
But people can always send patches. ;-)
--
Kaleb
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archiv
Kaleb S. KEITHLEY
2018-12-06 12:11:28 UTC
Permalink
Post by Kaleb S. KEITHLEY
Post by Kaleb S. KEITHLEY
And the other active maintainer (branto) and I don't have cycles to
devote to keeping (any part of) it building on 32-bit archs.
But people can always send patches. ;-)
If someone else would like to take over as maintainer, I'm happy to give
it up.

LMK.
--
Kaleb
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedor
Richard W.M. Jones
2018-12-10 18:22:17 UTC
Permalink
Rather predictably this has broken libvirt on i686 and armv7hl
(ie. 32 bit arches):

DEBUG util.py:439: - package libvirt-daemon-driver-storage-4.10.0-1.fc30.i686 requires libvirt-daemon-driver-storage-rbd = 4.10.0-1.fc30, but none of the providers can be installed
DEBUG util.py:439: - conflicting requests
DEBUG util.py:439: - nothing provides librados.so.2 needed by libvirt-daemon-driver-storage-rbd-4.10.0-1.fc30.i686

I filed a bug, against libvirt since it appears that no client
library for ceph will be forthcoming:

https://bugzilla.redhat.com/show_bug.cgi?id=1657928

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
_______________________________________________
devel mailing list -- ***@lists.fedoraproject.org
To unsubscribe send an email to devel-***@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/arc

Loading...