{"id":4779,"date":"2016-11-21T14:08:03","date_gmt":"2016-11-21T06:08:03","guid":{"rendered":"https:\/\/www.hmouse.tk\/?p=4779"},"modified":"2016-11-21T14:08:51","modified_gmt":"2016-11-21T06:08:51","slug":"hdfs%e7%bb%b4%e6%8a%a4%e6%89%8b%e5%86%8c","status":"publish","type":"post","link":"https:\/\/www.hmouse.cn\/?p=4779","title":{"rendered":"hdfs\u7ef4\u62a4\u624b\u518c"},"content":{"rendered":"

\u524d\u8bb0\uff1a\u6211\u4eec\u4e5f\u642d\u5efa\u4e86hdfs\uff0c\u6253\u7b97\u628a\u5176\u4e2d\u7684\u7ef4\u62a4\u7ecf\u5386\u8bb0\u5f55\u4e0b\u6765\uff0c\u4ee5\u65b9\u4fbf\u81ea\u5df1\u67e5\u627e\u5386\u53f2\u3002<\/p>\n

1\uff1a\u62a5\u9519\u65e0\u6cd5\u542f\u52a8namenode
2016-11-21 10:34:46,072 INFO org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode: Loading 177159 INodes.
2016-11-21 10:34:47,252 INFO org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf: Loaded FSImage in 1 seconds.
2016-11-21 10:34:47,253 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Loaded image for txid 454073842 from \/data\/hadoop\/hadoop\/tmp\/dfs\/name\/current\/fsimage_0000000000454073842
2016-11-21 10:34:47,258 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@32c8e539 expecting start txid #454073843
2016-11-21 10:34:47,260 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Start loading edits file http:\/\/hadoop3.i.zhihuishu.com:8480\/getJournal?jid=zhscluster&segmentTxId=454073896&storageInfo=-63%3A1565372169%3A0%3ACID-6393e2a5-76a8-42dd-9c37-95d951ef6f30<\/a>, http:\/\/hadoop1.i.zhihuishu.com:8480\/getJournal?jid=zhscluster&segmentTxId=454073896&storageInfo=-63%3A1565372169%3A0%3ACID-6393e2a5-76a8-42dd-9c37-95d951ef6f30<\/a>
2016-11-21 10:34:47,262 INFO org.apache.hadoop.hdfs.server.namenode.EditLogInputStream: Fast-forwarding stream ‘http:\/\/hadoop3.i.zhihuishu.com:8480\/getJournal?jid=zhscluster&segmentTxId=454073896&storageInfo=-63%3A1565372169%3A0%3ACID-6393e2a5-76a8-42dd-9c37-95d951ef6f30<\/a>, http:\/\/hadoop1.i.zhihuishu.com:8480\/getJournal?jid=zhscluster&segmentTxId=454073896&storageInfo=-63%3A1565372169%3A0%3ACID-6393e2a5-76a8-42dd-9c37-95d951ef6f30′<\/a> to transaction ID 454073843
2016-11-21 10:34:47,263 INFO org.apache.hadoop.hdfs.server.namenode.EditLogInputStream: Fast-forwarding stream ‘http:\/\/hadoop3.i.zhihuishu.com:8480\/getJournal?jid=zhscluster&segmentTxId=454073896&storageInfo=-63%3A1565372169%3A0%3ACID-6393e2a5-76a8-42dd-9c37-95d951ef6f30′<\/a> to transaction ID 454073843
2016-11-21 10:34:47,956 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception loading fsimage
java.io.IOException: There appears to be a gap in the edit log.  We expected txid 454073843, but got txid 454073896.
        at org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:215)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:584)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:644)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
2016-11-21 10:34:47,960 INFO org.mortbay.log: Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@hadoop1.i.zhihuishu.com:50070
2016-11-21 10:34:47,961 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode metrics system…
2016-11-21 10:34:47,962 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system stopped.
2016-11-21 10:34:47,962 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system shutdown complete.
2016-11-21 10:34:47,962 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
java.io.IOException: There appears to be a gap in the edit log.  We expected txid 454073843, but got txid 454073896.
        at org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:215)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:584)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:644)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
2016-11-21 10:34:47,964 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2016-11-21 10:34:47,971 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
\/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at rizhi143\/10.45.53.143
************************************************************\/
\u5904\u7406\u529e\u6cd5\uff1a\u7f51\u4e0a\u6709\u89e3\u51b3\u529e\u6cd5\uff0chttp:\/\/blog.csdn.net\/amber_amber\/article\/details\/46896719\u3002\u4f46\u662f\u6211\u6d4b\u8bd5\u7684\u65f6\u5019\u4e0d\u53ef\u4ee5\uff0c\u6240\u4ee5\u5168\u90e8\u683c\u5f0f\u5316\u4e86\u3002<\/a>
\u53c2\u8003url\uff1ahttps:\/\/my.oschina.net\/HIJAY\/blog\/220816<\/a>
1\u3001\u5220\u9664hdfs\/data\u4e2d\u7684\u6240\u6709\u6587\u4ef6
2\u3001bin\/stop-all.sh
3\u3001bin\/hadoop namenode -format
4\u3001bin\/start-all.sh<\/p>\n","protected":false},"excerpt":{"rendered":"

\u524d\u8bb0\uff1a\u6211\u4eec\u4e5f\u642d\u5efa\u4e86hdfs\uff0c\u6253\u7b97\u628a\u5176\u4e2d\u7684\u7ef4\u62a4\u7ecf\u5386\u8bb0\u5f55\u4e0b\u6765\uff0c\u4ee5\u65b9\u4fbf\u81ea\u5df1\u67e5\u627e\u5386\u53f2\u3002<\/p>\n

1\uff1a\u62a5\u9519\u65e0\u6cd5\u542f\u52a8namenode
2016-11-21 10:34:46,072 INFO org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode: Loading 177159 INodes.
2016-11-21 10:34:47,252 INFO org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf: Loaded FSImage in 1 seconds.
2016-11-21 10:34:47,253 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Loaded image for txid 454073842 from \/data\/hadoop\/hadoop\/tmp\/dfs\/name\/current\/fsimage_0000000000454073842
2016-11-21 10:34:47,258 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@32c8e539 expecting start txid #454073843
2016-11-21 10:34:47,260 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Start loading edits file http:\/\/hadoop3.i.zhihuishu.com:8480\/getJournal?jid=zhscluster&segmentTxId=454073896&storageInfo=-63%3A1565372169%3A0%3ACID-6393e2a5-76a8-42dd-9c37-95d951ef6f30<\/a>, http:\/\/hadoop1.i.zhihuishu.com:8480\/getJournal?jid=zhscluster&segmentTxId=454073896&storageInfo=-63%3A1565372169%3A0%3ACID-6393e2a5-76a8-42dd-9c37-95d951ef6f30<\/a>
2016-11-21 10:34:47,262 INFO org.apache.hadoop.hdfs.server.namenode.EditLogInputStream: Fast-forwarding stream ‘http:\/\/hadoop3.i.zhihuishu.com:8480\/getJournal?jid=zhscluster&segmentTxId=454073896&storageInfo=-63%3A1565372169%3A0%3ACID-6393e2a5-76a8-42dd-9c37-95d951ef6f30<\/a>, http:\/\/hadoop1.i.zhihuishu.com:8480\/getJournal?jid=zhscluster&segmentTxId=454073896&storageInfo=-63%3A1565372169%3A0%3ACID-6393e2a5-76a8-42dd-9c37-95d951ef6f30′<\/a> to transaction ID 454073843
2016-11-21 10:34:47,263 INFO org.apache.hadoop.hdfs.server.namenode.EditLogInputStream: Fast-forwarding stream ‘http:\/\/hadoop3.i.zhihuishu.com:8480\/getJournal?jid=zhscluster&segmentTxId=454073896&storageInfo=-63%3A1565372169%3A0%3ACID-6393e2a5-76a8-42dd-9c37-95d951ef6f30′<\/a> to transaction ID 454073843
2016-11-21 10:34:47,956 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception loading fsimage
java.io.IOException: There appears to be a gap in the edit log.  We expected txid 454073843, but got txid 454073896.
        at org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:215)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:584)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:644)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
2016-11-21 10:34:47,960 INFO org.mortbay.log: Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@hadoop1.i.zhihuishu.com:50070
2016-11-21 10:34:47,961 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode metrics system…
2016-11-21 10:34:47,962 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system stopped.
2016-11-21 10:34:47,962 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system shutdown complete.
2016-11-21 10:34:47,962 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
java.io.IOException: There appears to be a gap in the edit log.  We expected txid 454073843, but got txid 454073896.
        at org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:215)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:143)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:837)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:692)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:294)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:584)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:644)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
2016-11-21 10:34:47,964 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2016-11-21 10:34:47,971 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
\/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at rizhi143\/10.45.53.143
************************************************************\/
\u5904\u7406\u529e\u6cd5\uff1a\u7f51\u4e0a\u6709\u89e3\u51b3\u529e\u6cd5\uff0chttp:\/\/blog.csdn.net\/amber_amber\/article\/details\/46896719\u3002\u4f46\u662f\u6211\u6d4b\u8bd5\u7684\u65f6\u5019\u4e0d\u53ef\u4ee5\uff0c\u6240\u4ee5\u5168\u90e8\u683c\u5f0f\u5316\u4e86\u3002<\/a>
\u53c2\u8003url\uff1ahttps:\/\/my.oschina.net\/HIJAY\/blog\/220816<\/a>
1\u3001\u5220\u9664hdfs\/data\u4e2d\u7684\u6240\u6709\u6587\u4ef6
2\u3001bin\/stop-all.sh
3\u3001bin\/hadoop namenode -format
4\u3001bin\/start-all.sh<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[191,193],"class_list":["post-4779","post","type-post","status-publish","format-standard","hentry","category-5","tag-hadoop","tag-hdfs","category-5-id","post-seq-1","post-parity-odd","meta-position-corners","fix"],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/posts\/4779","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4779"}],"version-history":[{"count":1,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/posts\/4779\/revisions"}],"predecessor-version":[{"id":4780,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/posts\/4779\/revisions\/4780"}],"wp:attachment":[{"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4779"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4779"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4779"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}