- FreeBSD 11.3-STABLE/amd64 r354667 with five SSD da[2-6] in ZFS RAIDZ.
- Processes like 'zfs list -a -r -t all' periodically stall in 'D' state
- for long time (tens of seconds), pool has many snapshots.
- System runs several bhyve instances over ZVOLs.
- # gstat -aI3s -f 'da[2-6].*'
- dT: 3.000s w: 3.000s filter: da[2-6].*
- L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name
- 64 85 0 0 0.0 0 0 0.0 113.2| da2
- 64 85 0 0 0.0 0 0 0.0 112.8| da3
- 64 85 0 0 0.0 0 0 0.0 112.8| da4
- 64 64 0 0 0.0 0 0 0.0 97.0| da5
- 64 85 0 0 0.0 0 0 0.0 104.7| da6
- 64 85 0 0 0.0 0 0 0.0 112.8| da3.eli
- 64 85 0 0 0.0 0 0 0.0 112.8| da4.eli
- 64 85 0 0 0.0 0 0 0.0 113.2| da2.eli
- 64 64 0 0 0.0 0 0 0.0 97.0| da5.eli
- 64 85 0 0 0.0 0 0 0.0 104.7| da6.eli
- # zpool status -v sata
- pool: sata
- state: ONLINE
- scan: resilvered 58,0G in 0 days 00:08:02 with 0 errors on Wed Mar 13 21:34:28 2019
- config:
- NAME STATE READ WRITE CKSUM
- sata ONLINE 0 0 0
- raidz1-0 ONLINE 0 0 0
- da2.eli ONLINE 0 0 0
- da3.eli ONLINE 0 0 0
- da4.eli ONLINE 0 0 0
- da5.eli ONLINE 0 0 0
- da6.eli ONLINE 0 0 0
- logs
- raid/r0s3 ONLINE 0 0 0
- errors: No known data errors
- AVAGO MegaRAID SAS FreeBSD mrsas driver version: 07.709.04.00-fbsd
- mrsas0: <AVAGO Invader SAS Controller> port 0x4000-0x40ff mem 0xa0c00000-0xa0c0ffff,0xa0b00000-0xa0bfffff irq 32 at device 0.0 numa-domain 0 on pci5
- mrsas0: FW now in Ready state
- mrsas0: Using MSI-X with 32 number of vectors
- mrsas0: FW supports <96> MSIX vector,Online CPU 32 Current MSIX <32>
- mrsas0: max sge: 0x46, max chain frame size: 0x400, max fw cmd: 0x39f
- mrsas0: Issuing IOC INIT command to FW.
- mrsas0: IOC INIT response received from FW.
- mrsas0: System PD created target ID: 0x0
- mrsas0: System PD created target ID: 0x1
- mrsas0: System PD created target ID: 0x2
- mrsas0: System PD created target ID: 0x3
- mrsas0: System PD created target ID: 0x4
- mrsas0: System PD created target ID: 0x5
- mrsas0: System PD created target ID: 0x6
- mrsas0: FW supports: UnevenSpanSupport=1
- mrsas0: max_fw_cmds: 927 max_scsi_cmds: 911
- mrsas0: MSI-x interrupts setup success
- mrsas0: mrsas_ocr_thread
- da5 at mrsas0 bus 1 scbus17 target 5 lun 0
- da2 at mrsas0 bus 1 scbus17 target 2 lun 0
- da3 at mrsas0 bus 1 scbus17 target 3 lun 0
- da4 at mrsas0 bus 1 scbus17 target 4 lun 0
- da4: <ATA Micron_5200_MTFD U404> Fixed Direct Access SPC-4 SCSI device
- da4: Serial Number 18531FF58A9B
- da4: 150.000MB/s transfers
- da4: 3662830MB (7501476528 512 byte sectors)
- da2: <ATA Micron_5200_MTFD U404> Fixed Direct Access SPC-4 SCSI device
- da2: Serial Number 18532039E29C
- da2: 150.000MB/s transfers
- da2: 3662830MB (7501476528 512 byte sectors)
- da3: <ATA Micron_5200_MTFD U404> Fixed Direct Access SPC-4 SCSI device
- da3: Serial Number 18531FF58CF6
- da3: 150.000MB/s transfers
- da3: 3662830MB (7501476528 512 byte sectors)
- da5: <ATA Micron_5200_MTFD U404> Fixed Direct Access SPC-4 SCSI device
- da5: Serial Number 18531FF586C2
- da5: 150.000MB/s transfers
- da5: 3662830MB (7501476528 512 byte sectors)
- da6 at mrsas0 bus 1 scbus17 target 6 lun 0
- da6: <ATA XA3840LE10063 00ZU> Fixed Direct Access SPC-4 SCSI device
- da6: Serial Number HKT01DYR
- da6: 150.000MB/s transfers
- da6: 3662830MB (7501476528 512 byte sectors)
Raw Paste