{"id":4239,"date":"2014-12-03T16:44:24","date_gmt":"2014-12-03T08:44:24","guid":{"rendered":"https:\/\/www.hmouse.tk\/?p=4239"},"modified":"2015-06-07T11:00:39","modified_gmt":"2015-06-07T03:00:39","slug":"%e5%a4%a7%e6%95%b0%e6%8d%ae%e9%87%87%e9%9b%86%e4%b8%8e%e5%88%86%e6%9e%90","status":"publish","type":"post","link":"https:\/\/www.hmouse.cn\/?p=4239","title":{"rendered":"\u5927\u6570\u636e\u91c7\u96c6\u4e0e\u5206\u6790"},"content":{"rendered":"<p>\u6700\u8fd1\u516c\u53f8\u5728\u5f04\u5927\u6570\u636e\u7684\u91c7\u96c6\uff0c\u6709\u65f6\u95f4\u7684\u8bdd\u628a\u81ea\u5df1\u7684\u5fc3\u5f97\u4f53\u4f1a\u5199\u4e0b\uff0c\u7b97\u4f5c\u521d\u6b65\u8ba4\u8bc6\uff0c\u7b49\u4ee5\u540e\u89c1\u89e3\u66f4\u6df1\u7684\u65f6\u5019\uff0c\u518d\u770b\u8fd9\u7bc7\u6587\u7ae0\u5e94\u8be5\u522b\u6709\u4e00\u756a\u98ce\u5473\u3002<\/p>\n<p>\u516c\u53f8\u91c7\u7528\uff1a<\/p>\n<p>flume \u2013\u300bfdfs\u00a0\u00a0\u00a0 \uff08\u5143\u6570\u636e\u5b58\u50a8\uff09<\/p>\n<p>&#8211;\u300belasticsearch\u00a0\u00a0 \uff08\u7d22\u5f15\u5f0f\u5b58\u50a8\uff09<\/p>\n<p>&#8211;\u300bkafka\u00a0 &#8211;\u300b storm \u2013\u300b mysql \u6216 mongodb\u00a0\u00a0\u00a0 \uff08\u6570\u636e\u6d41\u5206\u6790\u4e0e\u5b58\u50a8\uff09<\/p>\n<p>\u5176\u5b9e\u6211\u5f53\u65f6\u5bf9storm\u7684\u4f5c\u7528\u5f88\u4e0d\u660e\u767d\uff0c\u56e0\u4e3aflume\u7406\u8bba\u4e0a\u53ef\u4ee5\u76f4\u63a5\u5199 mysql\u7684\uff0c\u4e2d\u95f4\u8fd9\u4e9b\u6b65\u9aa4\u7684\u7528\u9014\u662f\u4ec0\u4e48\uff1f\u662f\u4e0d\u662f\u592a\u7d2f\u8d58\u4e86\u3002\u540e\u6765\u4e00\u60f3\u662f\u57fa\u4e8e\u5927\u6570\u636e\u7684\uff0c\u5982\u679cmysql\u80fd\u5904\u7406\u4e0b\u6765\u7684\u8bdd\uff0c\u4f55\u5fc5\u9700\u8981hdfs\u3002\u5176\u5b9emysql\u5b58\u50a8\u7684\u662f\u5904\u7406\u540e\u7684\u6570\u636e\uff0c\u8b6c\u59825s\u7684pv\u7edf\u8ba1\uff0c\u522b\u4eba\u6709\u4e2a\u66f4\u597d\u7684storm\u7684\u4f5c\u7528\u56fe\u4e00\u770b\u4fbf\u77e5\uff1a<\/p>\n<p><a href=\"https:\/\/www.hmouse.cn\/wp-content\/uploads\/2014\/12\/storm.jpg\"><img loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;\" title=\"storm\u529f\u80fd\u56fe\" src=\"https:\/\/www.hmouse.cn\/wp-content\/uploads\/2014\/12\/storm_thumb.jpg\" alt=\"storm\u529f\u80fd\u56fe\" width=\"663\" height=\"433\" border=\"0\" \/><\/a><\/p>\n<p>\u53ef\u4ee5\u53d1\u73b0\u5728storm\u7684\u5904\u7406\u540e\u7684\u6570\u636e\u8d85\u7cbe\u7b80\u4e86\uff0c\u800c\u4fdd\u7559\u5728hdfs\u7684\u6570\u636e\u8fd8\u662f\u6709\u5fc5\u8981\u7684\uff0c\u56e0\u4e3astorm\u7684\u5904\u7406\u7684\u662f\u5df2\u77e5\u60c5\u51b5\uff0c\u53ef\u80fd\u9700\u8981\u518d\u6316\u6398\u7684\u8bdd\uff0c\u53ef\u4ee5\u901a\u8fc7mapreduce\u518d\u5206\u6790\uff0c\u6216\u8005\u505a\u6279\u91cf\u5904\u7406\uff08\u975e\u5b9e\u65f6\uff09\uff0c\u6211\u4eec\u5c31\u662f\u8fd9\u4e48\u505a\u7684\u3002<\/p>\n<p>\u5176\u5b9e\uff0c\u6709\u4e00\u4e9b\u516c\u53f8\u4e0d\u662f\u8fd9\u4e48\u505a\u7684\uff0c\u8b6c\u5982\u7f51\u6613\uff1a<\/p>\n<p><a title=\"http:\/\/www.infoq.com\/cn\/news\/2014\/04\/netease-spark-practice\" href=\"http:\/\/www.infoq.com\/cn\/news\/2014\/04\/netease-spark-practice\">http:\/\/www.infoq.com\/cn\/news\/2014\/04\/netease-spark-practice<\/a><\/p>\n<p>\u7528\u7684\u662fhdfs\uff08\u5b58\u50a8\uff09+ Spark + Hive \uff08\u540e\u97622\u4e2a\u505a\u5b9e\u65f6\u5904\u7406\uff09<\/p>\n<p>\u5176\u5b9e\u5c31\u662f Impala + Shark \u4ee3\u66ff\u4e86 kafka\u00a0 &#8211;\u300b storm \u2013\u300b mysql \u6216 mongodb\u00a0\u00a0 \uff0c2\u8005\u7684\u597d\u574f\uff0c\u4ece\u5904\u7406\u7684\u601d\u8def\u4e0a\u5c31\u4e0d\u4e00\u6837\uff0c\u4ee5\u540e\u53ef\u4ee5\u597d\u597d\u6d4b\u8bd5\u4e0b\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u6700\u8fd1\u516c\u53f8\u5728\u5f04\u5927\u6570\u636e\u7684\u91c7\u96c6\uff0c\u6709\u65f6\u95f4\u7684\u8bdd\u628a\u81ea\u5df1\u7684\u5fc3\u5f97\u4f53\u4f1a\u5199\u4e0b\uff0c\u7b97\u4f5c\u521d\u6b65\u8ba4\u8bc6\uff0c\u7b49\u4ee5\u540e\u89c1\u89e3\u66f4\u6df1\u7684\u65f6\u5019\uff0c\u518d\u770b\u8fd9\u7bc7\u6587\u7ae0\u5e94\u8be5\u522b\u6709\u4e00\u756a\u98ce\u5473\u3002<\/p>\n<p>\u516c\u53f8\u91c7\u7528\uff1a<\/p>\n<p>flume \u2013\u300bfdfs\u00a0\u00a0\u00a0 \uff08\u5143\u6570\u636e\u5b58\u50a8\uff09<\/p>\n<p>&#8211;\u300belasticsearch\u00a0\u00a0 \uff08\u7d22\u5f15\u5f0f\u5b58\u50a8\uff09<\/p>\n<p>&#8211;\u300bkafka\u00a0 &#8211;\u300b storm \u2013\u300b mysql \u6216 mongodb\u00a0\u00a0\u00a0 \uff08\u6570\u636e\u6d41\u5206\u6790\u4e0e\u5b58\u50a8\uff09<\/p>\n<p>\u5176\u5b9e\u6211\u5f53\u65f6\u5bf9storm\u7684\u4f5c\u7528\u5f88\u4e0d\u660e\u767d\uff0c\u56e0\u4e3aflume\u7406\u8bba\u4e0a\u53ef\u4ee5\u76f4\u63a5\u5199 mysql\u7684\uff0c\u4e2d\u95f4\u8fd9\u4e9b\u6b65\u9aa4\u7684\u7528\u9014\u662f\u4ec0\u4e48\uff1f\u662f\u4e0d\u662f\u592a\u7d2f\u8d58\u4e86\u3002\u540e\u6765\u4e00\u60f3\u662f\u57fa\u4e8e\u5927\u6570\u636e\u7684\uff0c\u5982\u679cmysql\u80fd\u5904\u7406\u4e0b\u6765\u7684\u8bdd\uff0c\u4f55\u5fc5\u9700\u8981hdfs\u3002\u5176\u5b9emysql\u5b58\u50a8\u7684\u662f\u5904\u7406\u540e\u7684\u6570\u636e\uff0c\u8b6c\u59825s\u7684pv\u7edf\u8ba1\uff0c\u522b\u4eba\u6709\u4e2a\u66f4\u597d\u7684storm\u7684\u4f5c\u7528\u56fe\u4e00\u770b\u4fbf\u77e5\uff1a<\/p>\n<p><a href=\"https:\/\/www.hmouse.cn\/wp-content\/uploads\/2014\/12\/storm.jpg\"><img loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;\" title=\"storm\u529f\u80fd\u56fe\" src=\"https:\/\/www.hmouse.cn\/wp-content\/uploads\/2014\/12\/storm_thumb.jpg\" alt=\"storm\u529f\u80fd\u56fe\" width=\"663\" height=\"433\" border=\"0\" \/><\/a><\/p>\n<p>\u53ef\u4ee5\u53d1\u73b0\u5728storm\u7684\u5904\u7406\u540e\u7684\u6570\u636e\u8d85\u7cbe\u7b80\u4e86\uff0c\u800c\u4fdd\u7559\u5728hdfs\u7684\u6570\u636e\u8fd8\u662f\u6709\u5fc5\u8981\u7684\uff0c\u56e0\u4e3astorm\u7684\u5904\u7406\u7684\u662f\u5df2\u77e5\u60c5\u51b5\uff0c\u53ef\u80fd\u9700\u8981\u518d\u6316\u6398\u7684\u8bdd\uff0c\u53ef\u4ee5\u901a\u8fc7mapreduce\u518d\u5206\u6790\uff0c\u6216\u8005\u505a\u6279\u91cf\u5904\u7406\uff08\u975e\u5b9e\u65f6\uff09\uff0c\u6211\u4eec\u5c31\u662f\u8fd9\u4e48\u505a\u7684\u3002<\/p>\n<p>\u5176\u5b9e\uff0c\u6709\u4e00\u4e9b\u516c\u53f8\u4e0d\u662f\u8fd9\u4e48\u505a\u7684\uff0c\u8b6c\u5982\u7f51\u6613\uff1a<\/p>\n<p><a title=\"http:\/\/www.infoq.com\/cn\/news\/2014\/04\/netease-spark-practice\" href=\"http:\/\/www.infoq.com\/cn\/news\/2014\/04\/netease-spark-practice\">http:\/\/www.infoq.com\/cn\/news\/2014\/04\/netease-spark-practice<\/a><\/p>\n<p>\u7528\u7684\u662fhdfs\uff08\u5b58\u50a8\uff09+ Spark + Hive \uff08\u540e\u97622\u4e2a\u505a\u5b9e\u65f6\u5904\u7406\uff09<\/p>\n<p>\u5176\u5b9e\u5c31\u662f Impala + Shark \u4ee3\u66ff\u4e86 kafka\u00a0 &#8211;\u300b storm \u2013\u300b mysql \u6216 mongodb\u00a0\u00a0 \uff0c2\u8005\u7684\u597d\u574f\uff0c\u4ece\u5904\u7406\u7684\u601d\u8def\u4e0a\u5c31\u4e0d\u4e00\u6837\uff0c\u4ee5\u540e\u53ef\u4ee5\u597d\u597d\u6d4b\u8bd5\u4e0b\u3002<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[110,109],"class_list":["post-4239","post","type-post","status-publish","format-standard","hentry","category-5","tag-storm","tag-109","category-5-id","post-seq-1","post-parity-odd","meta-position-corners","fix"],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/posts\/4239","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4239"}],"version-history":[{"count":4,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/posts\/4239\/revisions"}],"predecessor-version":[{"id":4379,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=\/wp\/v2\/posts\/4239\/revisions\/4379"}],"wp:attachment":[{"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4239"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4239"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.hmouse.cn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4239"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}