0

Our Goal is to use huge storage as 200TB for Kafka machine

In order to achieve this Goal , we create volume using LVM over 4 physical disks in RAID10 , when each disk size is 50T ,

Note - each disk as sdb/sdc/sdd/sde is actually RAID10 , when each RAID10 is holding 12 disks

In our kafka server we have 4 disks ( example from lsblk )

sdb                  8:16   0   50T  0 disk 
sdc                  8:32   0   50T  0 disk 
sdd                  8:48   0   50T  0 disk 
sde                  8:64   0   50T  0 disk

So we wrote the following procedure:

We combine the disks /dev/sdb and /dev/sdc. and /dev/sdd and /dev/sde , by adding them to the same volume group.- DB_vg

vgcreate DB_vg /dev/sdb /dev/sdc /dev/sdd /dev/sde

we create logical volumes ( we are using here 200T size )

lvcreate -L200T -n DB_lv DB_vg

We create the XFS filesystem

mkfs.ext4  -j -m 0  /dev/DB_vg/DB_lv -F

We create folder , in order to mount the LV to this folder

mkdir /var/lib/kafka_broker

mount /dev/DB_vg/DB_lv /var/lib/kafka_broker

from df -h , its should looks like this ( example )

/dev/mapper/DB_vg-DB_lv  200T   61M  200T   1%  /var/lib/kafka_broker

from lsblk

sdb                   
└─DB_vg-DB_lv 253:4    
sdc                   
└─DB_vg-DB_lv 253:4   
sdd                  
└─DB_vg-DB_lv 253:4    
sde                  
└─DB_vg-DB_lv 253:4

from blkid

blkid | grep LVM2_member
/dev/sda2: UUID="Y5MbyB-C5NN-hcPA-wd9R-jmdI-02ML-W9qIiu" TYPE="LVM2_member"    <-- sda2 is the OS
/dev/sdc: UUID="mtg8CY-b3j9-K1qZ-CRTV-SZum-gDx3-lbm4ZX" TYPE="LVM2_member"
/dev/sdb: UUID="MvfzqO-PV8N-dror-psl4-PfUf-7coa-6cb8lQ" TYPE="LVM2_member"
/dev/sdd: UUID="C4n63l-Uk3E-D65G-WcgI-xic2-cJLi-eSTUAa" TYPE="LVM2_member"
/dev/sde: UUID="d743Xp-eDxr-Dygk-HGgy-Dh9c-K3cx-kHtyqo" TYPE="LVM2_member"

So now we can use the LV /dev/mapper/DB_vg-DB_lv that mount to /var/lib/kafka_broker folder

and we can use the folder /var/lib/kafka_broker as storage with 200T for kafka broker machine

My Questions are:

  1. is it safe? to use huge storage that created by LVM using 4 Huge disks?

  2. can we create XFS filesystem when storage is around 200T size?

  3. am I right about my procedure or maybe I am missing something?

yael
  • 13,106
  • The disks as sdb/sdc/sdd/sde are new scratch disks – yael Nov 01 '23 at 19:00
  • in my procedure I not use pvcreate , shall I use it? – yael Nov 01 '23 at 19:33
  • You need the pvcreate before you can create an LVM volume group. I suggest that if you've not tried this before, then you set up some (small) scratch disks. See the first part of this answer – Chris Davies Nov 01 '23 at 19:48
  • ok, I accept your comment , second - do you think my Questions are reasonable enough? , or I need to add some additional info ? – yael Nov 01 '23 at 19:52
  • LVM used this way gives you one big filesystem, with zero load distribution. You can do that but I would prefer smaller volumes and let kafka itself handle the distribution. Kafka documentation even suggests letting it handle replication instead of relying on raid. https://kafka.apache.org/documentation/#diskandfs – frostschutz Nov 02 '23 at 07:13

1 Answers1

4

  1. As with any storage, this is as safe as your backups.

  2. XFS is supported up to 500TB, so this is fine.

  3. As far as I can tell, nothing is missing (vgcreate and vgextend can initialize PVs, so you don’t need to run pvcreate).

Stephen Kitt
  • 434,908
  • another thing if I can, as already mentioned each disk is actually represented the RAID10 , do you see a problem with that? ( I asked this because Kafka service need high disk performance ) – yael Nov 01 '23 at 20:11
  • 1
    I answered your questions, taking all the information therein into account. – Stephen Kitt Nov 01 '23 at 22:02
  • 1
    If the devices were not previously intialized as PVs with pvcreate(8), vgcreate will inititialize them, making them PVs. The pvcreate options for initializing devices are also available with vgcreate. – Stéphane Chazelas Nov 02 '23 at 06:15
  • 1
    @StéphaneChazelas wow, since October 2009! Thanks, I wasn’t aware of that! – Stephen Kitt Nov 02 '23 at 06:48