Mysql startet nach reboot nicht mehr

[Leagis]

Hallo

gestern Abend war der planmäßige Reboot meines Servers. Es wurden alle Dienste neugestartet bis auf mysql. Zu erst habe ich versucht mit

Code: Alles auswählen

service mysql start

zu starten. Als Ergebnis kam "failed". Danach habe ich mir meinen Error-Log angeschaut (/var/log/mysql.err)

Dies ist das Ergebnis:

Code: Alles auswählen

root@florian:/var/log# cat mysql.err
140209 00:47:57 mysqld_safe Starting mysqld daemon with databases from /var/lib/                               mysql
140209  0:47:57 [Note] Plugin 'FEDERATED' is disabled.
140209  0:47:57  InnoDB: Initializing buffer pool, size = 8.0M
140209  0:47:57  InnoDB: Completed initialization of buffer pool
InnoDB: Log scan progressed past the checkpoint lsn 6 1191961494
140209  0:47:57  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 6 1191961756
140209  0:47:57  InnoDB: Starting an apply batch of log records to the database.                               ..
InnoDB: Progress in percents: 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2                               5 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51                                52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78                                79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
140209  0:47:57  InnoDB: Error: the OS said file flush did not succeed
140209  0:47:57  InnoDB: Operating system error number 5 in a file operation.
InnoDB: Error number 5 means 'Input/output error'.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/operating-system-error-codes.html
InnoDB: File operation call: 'flush'.
InnoDB: Cannot continue operation.
140209 00:47:57 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ende                               d

Daraus habe ich erstmal auf einen Fehler der Festplatten und einen Smartctl durchgeführt. Hier ist das Ergebnis:

Code: Alles auswählen

root@proxmox:/home/sshclient# smartctl -H /dev/sda
smartctl 5.40 2010-07-12 r3124 [x86_64-unknown-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net
 
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
 
root@proxmox:/home/sshclient# smartctl -H /dev/sdb
smartctl 5.40 2010-07-12 r3124 [x86_64-unknown-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net
 
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
 
root@proxmox:/home/sshclient# smartctl -A /dev/sda
smartctl 5.40 2010-07-12 r3124 [x86_64-unknown-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net
 
=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   102   092   006    Pre-fail  Always       -       4257352
  3 Spin_Up_Time            0x0003   095   095   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       5
  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   081   060   030    Pre-fail  Always       -       141421627
  9 Power_On_Hours          0x0032   096   096   000    Old_age   Always       -       4047
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       5
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   001   001   000    Old_age   Always       -       434
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   056   052   045    Old_age   Always       -       44 (Lifetime Min/Max 30/48)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       4
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       8
194 Temperature_Celsius     0x0022   044   048   000    Old_age   Always       -       44 (0 29 0 0)
197 Current_Pending_Sector  0x0012   098   096   000    Old_age   Always       -       384
198 Offline_Uncorrectable   0x0010   098   096   000    Old_age   Offline      -       384
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       76987288784838
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       179644305426191
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       13373754141568
 
root@proxmox:/home/sshclient# smartctl -A /dev/sdb
smartctl 5.40 2010-07-12 r3124 [x86_64-unknown-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net
 
=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   103   095   006    Pre-fail  Always       -       6140056
  3 Spin_Up_Time            0x0003   093   093   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       10
  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   087   064   030    Pre-fail  Always       -       509392206
  9 Power_On_Hours          0x0032   081   081   000    Old_age   Always       -       17114
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       10
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   001   001   000    Old_age   Always       -       263
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
189 High_Fly_Writes         0x003a   099   099   000    Old_age   Always       -       1
190 Airflow_Temperature_Cel 0x0022   061   050   045    Old_age   Always       -       39 (Lifetime Min/Max 35/44)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       7
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       1222
194 Temperature_Celsius     0x0022   039   050   000    Old_age   Always       -       39 (0 15 0 0)
197 Current_Pending_Sector  0x0012   100   001   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   001   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       268225002619542
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       77739565324547
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       68143130391838

Jetzt bin ich aber leider etwas verloren und habe keine Ahnung, was ich mit den Ergebnissen anfangen kann. Was sind meine nächsten Schritte?

[DeletedUserReAsG]

Dateisystemfehler sollten auch nochmal explizit in den Systemlogs auftauchen. Mein Vorschlag bzgl. weiteren Vorgehens: Backup auf den aktuellen Stand bringen (soweit noch möglich), und Platte ersetzen.

[Leagis]

Danke für die Antwort

Kannst du mir erklären oder einen Link dazu geben, wie du den Status der Platten bewertet hast?

Meinst du mit dem Systemlog den kern.log ?

[DeletedUserReAsG]

Code: Alles auswählen

187 Reported_Uncorrect      0x0032   001   001   000    Old_age   Always       -       434

ist eigentlich deutlich.

Die Fehler sollten im syslog, in der messages, im kern.log, … zu finden sein, und etwa so aussehen, wie hier aus der messages eines kranken Servers:

Code: Alles auswählen

Feb  2 14:21:39 ks204143 kernel: [3449451.137383] sd 1:0:0:0: [sdb] Unhandled sense code
Feb  2 14:21:39 ks204143 kernel: [3449451.137385] sd 1:0:0:0: [sdb]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Feb  2 14:21:39 ks204143 kernel: [3449451.137389] sd 1:0:0:0: [sdb]  Sense Key : Medium Error [current] [descriptor]
Feb  2 14:21:39 ks204143 kernel: [3449451.137394] Descriptor sense data with sense descriptors (in hex):
Feb  2 14:21:39 ks204143 kernel: [3449451.137397]         72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00 
Feb  2 14:21:39 ks204143 kernel: [3449451.137406]         55 a6 40 f8 
Feb  2 14:21:39 ks204143 kernel: [3449451.137410] sd 1:0:0:0: [sdb]  Add. Sense: Unrecovered read error - auto reallocate failed
Feb  2 14:21:39 ks204143 kernel: [3449451.137415] sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 55 a6 40 58 00 01 00 00