Sharing hard drives with Ceph [feedly]

  

----
Sharing hard drives with Ceph
// Ceph

A group of users give hard drives to the system administrator of the Ceph cluster. In exchange, each of them get credentials to access a dedicated pool of a given size from the Ceph cluster.

The system administrator runs:

# ceph-authtool jean-keyring --create-keyring --name client.jean \     --gen-key --set-uid 458 \     --cap mon 'allow profile simple-rados-client' \     --cap osd 'allow rwx pool=jean-pool'  creating jean-keyring  # ceph auth import --in-file jean-keyring  imported keyring  # ceph auth get client.jean  exported keyring for client.jean  [client.jean]          key = AQCziVZT6EJoIRAA/HVxueyPmRGjvqQeOR40hQ==          auid = 458          caps mon = "allow profile simple-rados-client"          caps osd = "allow rwx pool=jean-pool"  

which creates the user client.jean in the Ceph cluster, with limited access to the monitors ( simple-rados-client ) and read write access to the OSDs but only for the benefit of accessing the (not yet existent), pool jean-pool. The pool is then created with:

# cat > create-pool-auid.py <<EOF  import rados  import sys  cluster = rados.Rados(conffile = '/etc/ceph/ceph.conf')  cluster.connect()  cluster.create_pool(sys.argv[1], int(sys.argv[2]))  EOF  # python create-pool-auid.py jean-pool 458  # ceph osd pool set-quota jean-pool max_bytes $((1024 * 1024 * 1024))  set-quota max_bytes = 1073741824 for pool jean-pool  

The python API is used to set jean as the owner of the pool, via the auid value 458 that was associated to it when it was created. The quota of the pool is set to 1GB and writes will fail when it is reached:

rados  put --pool jean-pool GROUP /etc/group  error putting jean-pool/GROUP: No space left on device  

The user is provided with the keyring that was just created and a ceph.conf file with the list of monitors to access the cluster:

[global]  auth_service_required = cephx  fsid = 8790ab57-f06f-4b27-8507-55c8d59e1327  auth_supported = cephx  auth_cluster_required = cephx  mon_host = 10.89.0.2  auth_client_required = cephx  

The user can then create an RBD volume with:

# rbd --name client.jean --keyring jean-keyring --pool jean-pool create --size 100 vda  # rbd --name client.jean --keyring jean-keyring --pool jean-pool info vda  rbd image 'vda':          size 102400 kB in 25 objects          order 22 (4096 kB objects)          block_name_prefix: rb.0.10f5.74b0dc51          format: 1  

It is then mapped as a block device with:

# rbd --name client.jean --keyring jean-keyring --pool jean-pool map vda  # dmesg  ...    232.099642] Key type ceph registered  [  232.099695] libceph: loaded (mon/osd proto 15/24)  [  232.100879] rbd: loaded rbd (rados block device)  [  232.102434] libceph: client4399 fsid 8790ab57-f06f-4b27-8507-55c8d59e1327  [  232.102971] libceph: mon0 10.89.0.2:6789 session established  [  232.159177]  rbd1: unknown partition table  [  232.159230] rbd: rbd1: added with size 0x6400000  # ls -l /dev/rbd1  brw-rw---- 1 root disk 251, 0 Apr 22 17:49 /dev/rbd1  

and can be formatted and mounted as a file system with:

# mkfs /dev/rbd1  mke2fs 1.42.9 (4-Feb-2014)  Filesystem label=  OS type: Linux  Block size=1024 (log=0)  Fragment size=1024 (log=0)  Stride=4096 blocks, Stripe width=4096 blocks  25688 inodes, 102400 blocks  5120 blocks (5.00%) reserved for the super user  First data block=1  Maximum filesystem blocks=67371008  13 block groups  8192 blocks per group, 8192 fragments per group  1976 inodes per group  Superblock backups stored on blocks:          8193, 24577, 40961, 57345, 73729  Allocating group tables: done  Writing inode tables: done  Writing superblocks and filesystem accounting information: done  # mount /dev/rbd1 /mnt  # df -h /mnt  Filesystem      Size  Used Avail Use% Mounted on  /dev/rbd1        97M  1.6M   91M   2% /mnt  

----

Shared via my feedly reader

Sent from my iPhone