{"id":1073181,"date":"2023-09-26T04:43:00","date_gmt":"2023-09-26T11:43:00","guid":{"rendered":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/?post_type=msr-blog-post&p=1073181"},"modified":"2024-09-25T03:46:11","modified_gmt":"2024-09-25T10:46:11","slug":"deepspeed4science","status":"publish","type":"msr-blog-post","link":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/articles\/deepspeed4science\/","title":{"rendered":"DeepSpeed4Science\uff1a\u5229\u7528\u5148\u8fdb\u7684AI\u7cfb\u7edf\u4f18\u5316\u6280\u672f\u5b9e\u73b0\u79d1\u5b66\u53d1\u73b0"},"content":{"rendered":"\n

\u672c\u6587\u8f6c\u8f7d\u81ea\u5fae\u8f6f DeepSpeed \u56e2\u961f\u5b98\u65b9\u77e5\u4e4e\u8d26\u53f7\uff1azhihu.com\/people\/deepspeed\uff0c\u7531\u5fae\u8f6f DeepSpeed \u56e2\u961f\u7ffb\u8bd1\u81ea\u5b98\u65b9\u82f1\u6587\u535a\u5ba2\uff1aAnnouncing the DeepSpeed4Science Initiative: Enabling large-scale scientific discovery through sophisticated AI system technologies\uff0c\u672c\u6587\u7565\u4f5c\u8c03\u6574\u3002<\/em><\/p>\n\n\n\n

\"\u88c5\u9970\u6027\u56fe\u7247\"<\/figure>\n\n\n\n

\u5728\u63a5\u4e0b\u6765\u7684\u5341\u5e74\u4e2d\uff0c\u6df1\u5ea6\u5b66\u4e60\u53ef\u80fd\u4f1a\u5f7b\u5e95\u6539\u53d8\u81ea\u7136\u79d1\u5b66\uff0c\u589e\u5f3a\u6211\u4eec\u5bf9\u81ea\u7136\u73b0\u8c61\u8fdb\u884c\u5efa\u6a21\u548c\u9884\u6d4b\u7684\u80fd\u529b\u3002\u8fd9\u53ef\u80fd\u9884\u793a\u7740\u79d1\u5b66\u63a2\u7d22\u7684\u65b0\u65f6\u4ee3\uff0c\u4e3a\u4ece\u836f\u7269\u5f00\u53d1\u5230\u53ef\u518d\u751f\u80fd\u6e90\u7684\u5404\u4e2a\u9886\u57df\u5e26\u6765\u91cd\u5927\u8fdb\u5c55\u3002\u4e3a\u4e86\u54cd\u5e94\u8fd9\u4e00\u673a\u4f1a\u4ee5\u53ca\u5fae\u8f6f\u201c\u4e88\u529b\u5168\u7403\u6bcf\u4e00\u4eba\u3001\u6bcf\u4e00\u7ec4\u7ec7\uff0c\u6210\u5c31\u4e0d\u51e1\u201d\u7684\u4f7f\u547d\uff0c\u5fae\u8f6f DeepSpeed \u56e2\u961f\u542f\u52a8\u4e86\u4e00\u4e2a\u540d\u4e3a DeepSpeed4Science \u7684\u65b0\u8ba1\u5212\uff0c\u65e8\u5728\u901a\u8fc7 AI \u7cfb\u7edf\u6280\u672f\u521b\u65b0\u5e2e\u52a9\u9886\u57df\u4e13\u5bb6\u89e3\u9501\u5f53\u4eca\u6700\u5927\u7684\u79d1\u5b66\u4e4b\u8c1c\u3002<\/p>\n\n\n\n

DeepSpeed \u7cfb\u7edf\u662f\u7531\u5fae\u8f6f\u5f00\u53d1\u7684\u4e1a\u754c\u9886\u5148\u7684\u5f00\u6e90 AI \u7cfb\u7edf\u6846\u67b6\uff0c\u5b83\u4e3a\u5404\u79cd AI \u786c\u4ef6\u4e0a\u7684\u6df1\u5ea6\u5b66\u4e60\u8bad\u7ec3\u548c\u63a8\u7406\u63d0\u4f9b\u4e86\u524d\u6240\u672a\u6709\u7684\u89c4\u6a21\u548c\u901f\u5ea6\u3002\u56fe1\u5c55\u793a\u4e86\u6211\u4eec\u5bf9 DeepSpeed4Science \u8fd9\u4e00\u65b0\u8ba1\u5212\u7684\u57fa\u672c\u65b9\u6cd5\u3002\u901a\u8fc7\u5229\u7528 DeepSpeed \u5f53\u524d\u7684\u6280\u672f\u65b9\u6848\uff08\u8bad\u7ec3\u3001\u63a8\u7406\u548c\u538b\u7f29\uff09\u4f5c\u4e3a\u57fa\u7840\u6280\u672f\u63a8\u52a8\u5668\uff0cDeepSpeed4Science \u5c06\u521b\u5efa\u4e00\u5957\u4e13\u4e3a\u52a0\u901f\u79d1\u5b66\u53d1\u73b0\u800c\u91cf\u8eab\u5b9a\u5236\u7684 AI \u7cfb\u7edf\u6280\u672f\uff0c\u4ee5\u5e94\u5bf9\u5176\u72ec\u7279\u7684\u590d\u6742\u6027\uff0c\u8d85\u8d8a\u7528\u4e8e\u52a0\u901f\u901a\u7528\u5927\u578b\u8bed\u8a00\u6a21\u578b\uff08LLMs\uff09\u7684\u5e38\u89c1\u6280\u672f\u65b9\u6cd5\u3002\u6211\u4eec\u4e0e\u62e5\u6709\u79d1\u5b66 AI \u6a21\u578b\u7684\u5185\u90e8\u548c\u5916\u90e8\u56e2\u961f\u7d27\u5bc6\u5408\u4f5c\uff0c\u4ee5\u53d1\u73b0\u548c\u89e3\u51b3\u9886\u57df\u7279\u5b9a AI \u7cfb\u7edf\u7684\u6311\u6218\u3002\u8fd9\u5305\u62ec\u6c14\u5019\u79d1\u5b66\u3001\u836f\u7269\u8bbe\u8ba1\u3001\u751f\u7269\u5b66\u7406\u89e3\u3001\u5206\u5b50\u52a8\u529b\u5b66\u6a21\u62df\u3001\u764c\u75c7\u8bca\u65ad\u548c\u76d1\u6d4b\u3001\u50ac\u5316\u5242\/\u6750\u6599\u53d1\u73b0\u3001\u548c\u5176\u4ed6\u9886\u57df\u3002<\/p>\n\n\n\n

\"\u56fe1\uff1aDeepSpeed4Science
\u56fe1\uff1aDeepSpeed4Science \u65b9\u6cd5\u6982\u8ff0\uff1a\u4e13\u4e3a\u52a0\u901f\u79d1\u5b66\u53d1\u73b0\u548c\u5e94\u5bf9\u5176\u590d\u6742\u6027\u800c\u91cf\u8eab\u5b9a\u5236\u7684 AI \u7cfb\u7edf\u6280\u672f\u5f00\u53d1\u3002<\/figcaption><\/figure>\n\n\n\n

<\/p>\n\n\n\n

\u6211\u4eec\u7684\u957f\u671f\u613f\u666f\u662f\u5c06 DeepSpeed4Science \u53d1\u5c55\u6210\u4e00\u4e2a\u7528\u4e8e\u5206\u4eab\u652f\u6301\u79d1\u5b66\u53d1\u73b0\u7684\u5148\u8fdb AI \u6280\u672f\u7684\u8f6f\u4ef6\u5e73\u53f0\u548c\u7edf\u4e00\u4ee3\u7801\u4ed3\u5e93\u3002DeepSpeed4Science \u7684\u8bbe\u8ba1\u65e8\u5728\u5305\u5bb9\u6027\uff0c\u547c\u5e94\u5fae\u8f6f\u7684\u201cAI for Good\u201d\u627f\u8bfa\u3002\u8fd9\u4f53\u73b0\u5728\u8be5\u8ba1\u5212\u5bf9\u4e00\u7cfb\u5217\u6807\u5fd7\u6027\u79d1\u5b66\u6a21\u578b\u7684\u652f\u6301\u4e0a\uff0c\u4ed6\u4eec\u4ee3\u8868\u4e86\u4e00\u4e9b\u6700\u5173\u952e\u7684 AI4Science \u5e94\u7528\u573a\u666f\u3002\u5728\u8fd9\u7bc7\u535a\u5ba2\u4e2d\uff0c\u6211\u4eec\u5c55\u793a\u4e86 DeepSpeed4Science \u5982\u4f55\u5e2e\u52a9\u89e3\u51b3\u7ed3\u6784\u751f\u7269\u5b66\u7814\u7a76\u4e2d\u7684\u4e24\u4e2a\u5173\u952e AI \u7cfb\u7edf\u6311\u6218\uff1a(1) \u89e3\u51b3\u4e86\u4ee5 Evoformer \u4e3a\u4e2d\u5fc3\u7684\u86cb\u767d\u8d28\u7ed3\u6784\u9884\u6d4b\u6a21\u578b\u4e2d\u7684\u5185\u5b58\u7206\u70b8\u95ee\u9898\uff0c\u4ee5\u53ca(2)\u4e3a\u66f4\u597d\u5730\u7406\u89e3\u5f15\u53d1\u5927\u6d41\u884c\u7684\u75c5\u6bd2\u7684\u8fdb\u5316\u63d0\u4f9b AI \u6a21\u578b\u957f\u5e8f\u5217\u652f\u6301\u3002<\/p>\n\n\n\n

\u6211\u4eec\u7684\u521d\u671f\u4e3b\u8981\u5408\u4f5c\u8005<\/h2>\n\n\n\n

DeepSpeed4Science \u7684\u65b0\u7cfb\u7edf\u6280\u672f\u53ef\u4ee5\u7528\u4e8e\u5f88\u591a\u63a8\u52a8\u79d1\u5b66\u8fb9\u754c\u7684\u6807\u5fd7\u6027\u6a21\u578b\uff0c\u8d4b\u80fd AI \u9a71\u52a8\u7684\u79d1\u5b66\u53d1\u73b0\u3002\u76ee\u524d\uff0cDeepSpeed4Science \u5f88\u8363\u5e78\u5730\u652f\u6301\u6765\u81ea\u5fae\u8f6f\u7814\u7a76\u9662 AI4Science\uff08\u5fae\u8f6f\u7814\u7a76\u9662\u79d1\u5b66\u667a\u80fd\u4e2d\u5fc3\uff09\u3001\u5fae\u8f6f WebXT\/Bing\u3001\u7f8e\u56fd\u80fd\u6e90\u90e8\u56fd\u5bb6\u5b9e\u9a8c\u5ba4\u548c\u591a\u6240\u5927\u5b66\u7684\u51e0\u4e2a\u5173\u952e\u79d1\u5b66\u6a21\u578b\u3002<\/p>\n\n\n\n

\u5fae\u8f6f\u5185\u90e8\u5408\u4f5c\u4f19\u4f34<\/h3>\n\n\n\n
    \n
  • \u79d1\u5b66\u57fa\u7840\u6a21\u578b\uff08Scientific Foundation Model\uff0cSFM\uff09\uff0c\u5fae\u8f6f\u7814\u7a76\u9662 AI4Science<\/li>\n<\/ul>\n\n\n\n
    \"diagram\"<\/figure>\n\n\n\n
    \"\u56fe2\uff1a\u79d1\u5b66\u57fa\u7840\u6a21\u578b\uff08Scientific
    \u56fe2\uff1a\u79d1\u5b66\u57fa\u7840\u6a21\u578b\uff08Scientific Foundation Model\uff0cSFM\uff09\u53ca\u5176\u5f53\u524d\u63a2\u7d22\uff1aDistributional Graphormer\u3002<\/figcaption><\/figure>\n\n\n\n

    \u79d1\u5b66\u57fa\u7840\u6a21\u578b\uff08SFM\uff09\u65e8\u5728\u521b\u5efa\u4e00\u4e2a\u5927\u89c4\u6a21\u57fa\u7840\u6a21\u578b\uff0c\u4ee5\u652f\u6301\u81ea\u7136\u79d1\u5b66\u53d1\u73b0\u3002\u5176\u652f\u6301\u591a\u79cd\u8f93\u5165\u6a21\u6001\u3001\u591a\u4e2a\u79d1\u5b66\u9886\u57df\uff08\u4f8b\u5982\uff0c\u836f\u7269\u3001\u6750\u6599\u3001\u751f\u7269\u5b66\u3001\u5065\u5eb7\u7b49\uff09\u7684\u8ba1\u7b97\u4efb\u52a1\u3002DeepSpeed4Science \u5408\u4f5c\u4f19\u4f34\u5173\u7cfb\u5c06\u4e3a SFM \u56e2\u961f\u63d0\u4f9b\u65b0\u7684\u8bad\u7ec3\u548c\u63a8\u7406\u6280\u672f\uff0c\u4ee5\u52a9\u529b\u4ed6\u4eec\u7684\u65b0 AI \u65b9\u6cd5\uff08\u4f8b\u5982Distributional Graphormer\uff09\uff0c\u4e3a\u5176\u5e26\u6765\u66f4\u9ad8\u7684\u6548\u7387\u4e0e\u66f4\u591a\u7684\u7814\u7a76\u7a81\u7834\u3002<\/p>\n\n\n\n

      \n
    • ClimaX\uff0c\u5fae\u8f6f\u7814\u7a76\u9662 AI4Science<\/li>\n<\/ul>\n\n\n\n
      \"\u56fe3\uff1aClimaX
      \u56fe3\uff1aClimaX \u662f\u7b2c\u4e00\u4e2a\u8bbe\u8ba1\u7528\u4e8e\u6267\u884c\u5404\u79cd\u5929\u6c14\u548c\u6c14\u5019\u5efa\u6a21\u4efb\u52a1\u7684\u57fa\u7840\u6a21\u578b<\/figcaption><\/figure>\n\n\n\n

      <\/p>\n\n\n\n

      \u6211\u4eec\u7684\u6c14\u5019\u6b63\u5728\u53d1\u751f\u53d8\u5316\uff0c\u5bfc\u81f4\u6781\u7aef\u5929\u6c14\u4e8b\u4ef6\u7684\u9891\u7387\u589e\u52a0\u3002\u4e3a\u4e86\u51cf\u8f7b\u8d1f\u9762\u5f71\u54cd\uff0c\u9884\u6d4b\u8fd9\u4e9b\u4e8b\u4ef6\u5c06\u53d1\u751f\u7684\u5730\u65b9\u53d8\u5f97\u8d8a\u6765\u8d8a\u91cd\u8981\u3002ClimaX \u662f\u7b2c\u4e00\u4e2a\u8bbe\u8ba1\u7528\u4e8e\u6267\u884c\u5404\u79cd\u5929\u6c14\u548c\u6c14\u5019\u5efa\u6a21\u4efb\u52a1\u7684\u57fa\u7840\u6a21\u578b\u3002\u5b83\u53ef\u4ee5\u5438\u6536\u8bb8\u591a\u5177\u6709\u4e0d\u540c\u53d8\u91cf\u548c\u5206\u8fa8\u7387\u7684\u6570\u636e\u96c6\u4ee5\u63d0\u9ad8\u5929\u6c14\u9884\u62a5\u7684\u51c6\u786e\u6027\u3002DeepSpeed4Science \u6b63\u5728\u4e3a ClimaX \u521b\u5efa\u65b0\u7684\u7cfb\u7edf\u652f\u6301\u548c\u52a0\u901f\u7b56\u7565\uff0c\u4ee5\u9ad8\u6548\u5730\u9884\u8bad\u7ec3\/\u5fae\u8c03\u66f4\u5927\u7684\u57fa\u7840\u6a21\u578b\uff0c\u540c\u65f6\u5904\u7406\u975e\u5e38\u5927\u7684\u9ad8\u5206\u8fa8\u7387\u56fe\u50cf\u6570\u636e\uff08\u4f8b\u5982\uff0c\u6570\u5341\u5230\u6570\u767ePB\uff09\u548c\u957f\u5e8f\u5217\u3002<\/p>\n\n\n\n

        \n
      • \u5206\u5b50\u52a8\u529b\u5b66\u4e0e\u673a\u5668\u5b66\u4e60\u529b\u573a \uff08Molecular Dynamics and Machine Learning Force Field\uff09\uff0c\u5fae\u8f6f\u7814\u7a76\u9662 AI4Science<\/li>\n<\/ul>\n\n\n\n
        \"\u56fe4\uff1a\u4e00\u767e\u4e07\u6b65\u7684\u5206\u5b50\u52a8\u529b\u5b66\u6a21\u62df\uff1aRBD-\u86cb\u767d\uff08RBD-protein\uff09\u4e0e\u86cb\u767d\u6291\u5236\u5242\uff08protein
        \u56fe4\uff1a\u4e00\u767e\u4e07\u6b65\u7684\u5206\u5b50\u52a8\u529b\u5b66\u6a21\u62df\uff1aRBD-\u86cb\u767d\uff08RBD-protein\uff09\u4e0e\u86cb\u767d\u6291\u5236\u5242\uff08protein inhibitor\uff09\u76f8\u4e92\u4f5c\u7528\u3002<\/figcaption><\/figure>\n\n\n\n

        <\/p>\n\n\n\n

        \u8fd9\u4e2a\u9879\u76ee\u6a21\u62df\u4e86\u4f7f\u7528 AI \u9a71\u52a8\u7684\u529b\u573a\u6a21\u578b\u8fdb\u884c\u8fd1\u4f3c\u7b2c\u4e00\u6027\u539f\u7406\u8ba1\u7b97\u7cbe\u5ea6\u7684\u5927\u578b\uff08\u767e\u4e07\u539f\u5b50\uff09\u5206\u5b50\u7cfb\u7edf\u7684\u52a8\u6001\u6a21\u62df\uff0c\u540c\u65f6\u4fdd\u6301\u4e86\u7ecf\u5178\u5206\u5b50\u52a8\u529b\u5b66\u7684\u6548\u7387\u548c\u53ef\u6269\u5c55\u6027\u3002\u8fd9\u4e9b\u6a21\u62df\u8db3\u591f\u9ad8\u6548\uff0c\u53ef\u4ee5\u751f\u6210\u8db3\u591f\u957f\u7684\u8f68\u8ff9\u6765\u89c2\u5bdf\u5316\u5b66\u4e0a\u6709\u610f\u4e49\u7684\u4e8b\u4ef6\u3002\u901a\u5e38\uff0c\u8fd9\u4e2a\u8fc7\u7a0b\u9700\u8981\u6570\u767e\u4e07\u81f3\u6570\u5341\u4ebf\u7684\u63a8\u7406\u6b65\u9aa4\u3002\u8fd9\u5bf9\u4f18\u5316\u56fe\u795e\u7ecf\u7f51\u7edc\uff08GNN\uff09+ LLM \u6a21\u578b\u7684\u63a8\u7406\u901f\u5ea6\u63d0\u51fa\u4e86\u91cd\u5927\u6311\u6218\uff0cDeepSpeed4Science \u5c06\u4e3a\u6b64\u63d0\u4f9b\u65b0\u7684\u52a0\u901f\u7b56\u7565\u3002<\/p>\n\n\n\n

          \n
        • \u5fae\u8f6f\u5929\u6c14\uff0c\u5fae\u8f6fWebXT\/Bing<\/li>\n<\/ul>\n\n\n\n

          <\/p>\n\n\n\n

          \"\u56fe5\uff1a\u5fae\u8f6f\u964d\u6c34\u9884\u62a5\uff08\u6bcf4\u5206\u949f\u4e00\u6b21\u5bf9\u63a5\u4e0b\u67654\u5c0f\u65f6\u8fdb\u884c\u9884\u6d4b\uff09\u3002\"
          \u56fe5\uff1a\u5fae\u8f6f\u964d\u6c34\u9884\u62a5\uff08\u6bcf4\u5206\u949f\u4e00\u6b21\u5bf9\u63a5\u4e0b\u67654\u5c0f\u65f6\u8fdb\u884c\u9884\u6d4b\uff09\u3002<\/figcaption><\/figure>\n\n\n\n

          \u5fae\u8f6f\u5929\u6c14\u63d0\u4f9b\u7cbe\u786e\u7684\u5929\u6c14\u4fe1\u606f\uff0c\u5e2e\u52a9\u7528\u6237\u4e3a\u4ed6\u4eec\u7684\u751f\u6d3b\u65b9\u5f0f\u3001\u5065\u5eb7\u3001\u5de5\u4f5c\u548c\u6d3b\u52a8\u505a\u51fa\u66f4\u597d\u7684\u51b3\u7b56\u2014\u2014\u5305\u62ec\u6bcf\u5c0f\u65f6\u591a\u6b21\u66f4\u65b0\u7684\u51c6\u786e\u768410\u5929\u5168\u7403\u5929\u6c14\u9884\u62a5\u3002\u6b64\u524d\uff0c\u5fae\u8f6f\u5929\u6c14\u53d7\u76ca\u4e8e DeepSpeed \u6280\u672f\uff0c\u52a0\u901f\u4e86\u4ed6\u4eec\u7684\u591a GPU \u8bad\u7ec3\u73af\u5883\u3002\u76ee\u524d\uff0cDeepSpeed4Science \u6b63\u5728\u4e0e\u5fae\u8f6f WebXT \u5929\u6c14\u9884\u62a5\u56e2\u961f\u5408\u4f5c\uff0c\u8fdb\u4e00\u6b65\u589e\u5f3a\u5fae\u8f6f\u5929\u6c14\u9884\u62a5\u670d\u52a1\u7684\u6700\u65b0\u529f\u80fd\u548c\u6539\u8fdb\u3002<\/p>\n\n\n\n

          \u5916\u90e8\u5408\u4f5c\u8005<\/h3>\n\n\n\n

          DeepSpeed4Science \u7684\u65c5\u7a0b\u59cb\u4e8e\u4e24\u4e2a\u5f00\u521b\u6027\u7684\u57fa\u4e8e LLM \u7684\u7ed3\u6784\u751f\u7269\u5b66\u7814\u7a76 AI \u6a21\u578b\uff1a\u6765\u81ea\u54e5\u4f26\u6bd4\u4e9a\u5927\u5b66\u7684 OpenFold\uff0c\u4e00\u4e2a\u5f00\u6e90\u7684\u9ad8\u4fdd\u771f\u86cb\u767d\u8d28\u7ed3\u6784\u9884\u6d4b\u6a21\u578b\uff1b\u4ee5\u53ca\u6765\u81ea\u963f\u8d21\u56fd\u5bb6\u5b9e\u9a8c\u5ba4\u7684 GenSLMs\uff0c\u4e00\u4e2a\u83b7\u5f97 ACM \u6208\u767b\u8d1d\u5c14\u5956\u7684\u7528\u4e8e\u5b66\u4e60 SARS-CoV-2\uff08COVID-19\uff09 \u57fa\u56e0\u7ec4\u7684\u8fdb\u5316\u7684\u8bed\u8a00\u6a21\u578b\u3002\u4f5c\u4e3a\u6b64\u6b21\u53d1\u5e03\u7684\u7279\u8272\u5c55\u793a\uff0c\u5b83\u4eec\u4ee3\u8868\u4e86\u5f53\u4eca AI \u9a71\u52a8\u7684\u7ed3\u6784\u751f\u7269\u5b66\u7814\u7a76\u9762\u4e34\u7684\u4e24\u4e2a\u5e38\u89c1 AI \u7cfb\u7edf\u6311\u6218\u3002\u6211\u4eec\u5c06\u5728\u4e0b\u4e00\u8282\u4e2d\u8ba8\u8bba DeepSpeed4Science \u5982\u4f55\u8d4b\u80fd\u8fd9\u4e9b\u79d1\u5b66\u7814\u7a76\u3002<\/p>\n\n\n\n

          \u6b64\u5916\uff0cDeepSpeed4Science \u6700\u8fd1\u6269\u5927\u4e86\u5176\u8303\u56f4\uff0c\u4ee5\u652f\u6301\u66f4\u591a\u6837\u7684\u79d1\u5b66\u6a21\u578b\u3002\u4f8b\u5982\uff0c\u5728\u6211\u4eec\u4e0e\u963f\u8d21\u56fd\u5bb6\u5b9e\u9a8c\u5ba4\u5408\u4f5c\u8bad\u7ec3 Aurora Exascale \u7cfb\u7edf\u4e0a\u7684\u4e07\u4ebf\u53c2\u6570\u79d1\u5b66\u6a21\u578b\u7684\u5de5\u4f5c\u4e2d\uff0cDeepSpeed4Science \u6280\u672f\u5c06\u5e2e\u52a9\u4ed6\u4eec\u8fbe\u5230\u8fd9\u4e00\u5173\u952e\u4efb\u52a1\u6240\u9700\u7684\u6027\u80fd\u8981\u6c42\u548c\u53ef\u6269\u5c55\u6027\u3002\u6b64\u5916\uff0c\u901a\u8fc7\u4e0e\u6a61\u6811\u5cad\u56fd\u5bb6\u5b9e\u9a8c\u5ba4\u548c\u56fd\u5bb6\u764c\u75c7\u7814\u7a76\u6240\uff08NCI\uff09\u5408\u4f5c\u8fdb\u884c\u764c\u75c7\u76d1\u6d4b\uff0cDeepSpeed4Science \u5c06\u5e2e\u52a9\u4ece\u975e\u7ed3\u6784\u5316\u7684\u4e34\u5e8a\u6587\u672c\u4e2d\u9ad8\u4fdd\u771f\u5730\u63d0\u53d6\u548c\u5206\u7c7b\u4fe1\u606f\uff0c\u4ee5\u4f9b MOSSAIC \u9879\u76ee\u4f7f\u7528\u3002Brookhaven \u56fd\u5bb6\u5b9e\u9a8c\u5ba4\u8fd8\u5c06\u91c7\u7528 DeepSpeed4Science \u6280\u672f\uff0c\u652f\u6301\u4f7f\u7528 LLMs \u5f00\u53d1\u5927\u578b\u6570\u5b57\u53cc\u80de\u80ce\u6a21\u578b\uff0c\u4ee5\u4fbf\u4e3a\u6e05\u6d01\u80fd\u6e90\u7814\u7a76\u4ea7\u751f\u66f4\u771f\u5b9e\u7684\u6a21\u62df\u6570\u636e\u3002\u60a8\u53ef\u4ee5\u5728 deepspeed4science.ai \u4e0a\u627e\u5230\u6709\u5173\u6211\u4eec\u5916\u90e8\u5408\u4f5c\u8005\u53ca\u5176\u79d1\u5b66\u4efb\u52a1\u7684\u66f4\u591a\u8be6\u7ec6\u4fe1\u606f\u3002<\/p>\n\n\n\n

          \u5408\u4f5c\u5c55\u793a<\/h2>\n\n\n\n

          \u5c55\u793a\uff08I\uff09\uff1aDeepSpeed4Science \u901a\u8fc7 DS4Sci_EvoformerAttention \u6d88\u9664\u4ee5 Evoformer \u4e3a\u4e2d\u5fc3\u7684\u7ed3\u6784\u751f\u7269\u5b66\u6a21\u578b\u7684\u5185\u5b58\u7206\u70b8\u95ee\u9898<\/strong><\/p>\n\n\n\n

          \"\u56fe6\uff1a\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d<\/figure>\n\n\n\n
          \"\u56fe6\uff1a\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d
          \u56fe6\uff1a\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d OpenFold \u5bf9 PDB \u94fe7B3A_A\u7684\u9884\u6d4b\u3002<\/figcaption><\/figure>\n\n\n\n

          OpenFold \u662f DeepMind \u7684 AlphaFold2 \u7684\u5f00\u6e90\u793e\u533a\u518d\u73b0\uff0c\u4f7f\u5176\u53ef\u4ee5\u5728\u65b0\u6570\u636e\u96c6\u4e0a\u8bad\u7ec3\u6216\u5fae\u8c03 AlphaFold2\u3002\u7814\u7a76\u4eba\u5458\u5df2\u7ecf\u4f7f\u7528\u5b83\u4ece\u5934\u5f00\u59cb\u91cd\u65b0\u8bad\u7ec3 AlphaFold2\uff0c\u751f\u6210\u65b0\u7684\u6a21\u578b\u53c2\u6570\u96c6\uff0c\u7814\u7a76 AlphaFold2 \u7684\u65e9\u671f\u8bad\u7ec3\u9636\u6bb5\uff08\u56fe6\uff09\uff0c\u5e76\u5f00\u53d1\u65b0\u7684\u86cb\u767d\u8d28\u6298\u53e0\u7cfb\u7edf\u3002<\/p>\n\n\n\n

          <\/p>\n\n\n\n

          \"\u56fe7\uff1a\u5728
          \u56fe7\uff1a\u5728 OpenFold \u4e2d\uff0c\u5bf9\u591a\u5e8f\u5217\u6bd4\u5bf9\uff08MSA\uff09Attention \u5185\u6838\uff08\u5305\u542b\u504f\u5dee\uff09\u53d8\u4f53\u7684\u8bad\u7ec3\u5cf0\u503c\u5185\u5b58\u9700\u6c42\u3002DeepSpeed4Science \u7684\u4e00\u79cd\u65b0\u89e3\u51b3\u65b9\u6848 DS4Sci_EvoformerAttention \u5728\u4e0d\u5f71\u54cd\u6a21\u578b\u54c1\u8d28\u7684\u6761\u4ef6\u4e0b\u663e\u8457\u5730\u51cf\u5c11\u4e86 OpenFold \u7684\u8bad\u7ec3\u5cf0\u503c\u5185\u5b58\u9700\u6c42\uff08\u6700\u591a13\u500d\uff09\u3002<\/figcaption><\/figure>\n\n\n\n

          \u5c3d\u7ba1 OpenFold \u6709\u4f7f\u7528\u6700\u5148\u8fdb\u7684\u7cfb\u7edf\u6280\u672f\u8fdb\u884c\u6027\u80fd\u548c\u5185\u5b58\u4f18\u5316\uff0c\u4f46\u4ece\u5934\u5f00\u59cb\u8bad\u7ec3 AlphaFold2 \u4ecd\u7136\u5728\u8ba1\u7b97\u4e0a\u5f88\u6602\u8d35\u3002\u76ee\u524d\u9636\u6bb5\u7684\u6a21\u578b\u53c2\u6570\u5f88\u5c0f\uff0c\u53ea\u67099300\u4e07\u4e2a\u53c2\u6570\uff0c\u4f46\u5b83\u5305\u542b\u4e86\u51e0\u4e2a\u9700\u8981\u975e\u5e38\u5927\u7684\u4e2d\u95f4\u5185\u5b58\u7684\u7279\u6b8a Attention \u53d8\u4f53\u3002\u5728\u6807\u51c6 AlphaFold2 \u8bad\u7ec3\u7684\u201c\u5fae\u8c03\u201d\u9636\u6bb5\uff0c\u53ea\u662f\u8fd9\u4e9b\u53d8\u4f53\u4e2d\u7684\u5176\u4e2d\u4e00\u4e2a\u5728\u534a\u7cbe\u5ea6\u4e0b\u5c31\u751f\u6210\u4e86\u8d85\u8fc7 12GB \u7684\u5f20\u91cf\uff0c\u4f7f\u5176\u5cf0\u503c\u5185\u5b58\u8981\u6c42\u8fdc\u8fdc\u8d85\u8fc7\u4e86\u76f8\u540c\u5927\u5c0f\u7684\u8bed\u8a00\u6a21\u578b\u3002\u5373\u4f7f\u4f7f\u7528\u50cf activation checkpointing \u548c DeepSpeed ZeRO \u4f18\u5316\u8fd9\u6837\u7684\u6280\u672f\uff0c\u8fd9\u79cd\u5185\u5b58\u7206\u70b8\u95ee\u9898\u4ecd\u7136\u4e25\u91cd\u9650\u5236\u4e86\u53ef\u8bad\u7ec3\u6a21\u578b\u7684\u5e8f\u5217\u957f\u5ea6\u548c MSA \u6df1\u5ea6\u3002\u6b64\u5916\uff0c\u8fd1\u4f3c\u7b56\u7565\u53ef\u80fd\u4f1a\u663e\u8457\u5f71\u54cd\u6a21\u578b\u7684\u51c6\u786e\u6027\u548c\u6536\u655b\u6027\uff0c\u540c\u65f6\u4ecd\u7136\u5bfc\u81f4\u5185\u5b58\u7206\u70b8\uff0c\u5982\u56fe7\u5de6\u4fa7\uff08\u6a59\u8272\uff09\u6240\u793a\u3002<\/p>\n\n\n\n

          \u4e3a\u4e86\u5e94\u5bf9\u7ed3\u6784\u751f\u7269\u5b66\u7814\u7a76\uff08\u4f8b\u5982\uff0c\u86cb\u767d\u8d28\u7ed3\u6784\u9884\u6d4b\u548c\u5e73\u8861\u5206\u5e03\u9884\u6d4b\uff09\u4e2d\u7684\u8fd9\u4e00\u5e38\u89c1\u7cfb\u7edf\u6311\u6218\uff0cDeepSpeed4Science \u901a\u8fc7\u4e3a\u8fd9\u7c7b\u79d1\u5b66\u6a21\u578b\u4e2d\u5e7f\u6cdb\u51fa\u73b0\u7684\u6ce8\u610f\u529b\u53d8\u4f53\uff08\u5373 EvoformerAttention\uff09\u8bbe\u8ba1\u5b9a\u5236\u7684\u7cbe\u786e\u6ce8\u610f\u529b\u5185\u6838\u6765\u89e3\u51b3\u8fd9\u4e00\u5185\u5b58\u6548\u7387\u95ee\u9898\u3002\u5177\u4f53\u6765\u8bf4\uff0c\u6211\u4eec\u8bbe\u8ba1\u4e86\u4e00\u5957\u7531\u590d\u6742\u7684\u878d\u5408\/\u77e9\u9635\u5206\u5757\u7b56\u7565\u548c\u52a8\u6001\u5185\u5b58\u51cf\u5c11\u65b9\u6cd5\u800c\u7ec4\u6210\u7684\u9ad8\u5185\u5b58\u6548\u7387 DS4Sci_EvoformerAttention \u5185\u6838\uff0c\u4f5c\u4e3a\u9ad8\u8d28\u91cf\u673a\u5668\u5b66\u4e60\u6a21\u5757\u4f9b\u66f4\u5e7f\u6cdb\u7684\u751f\u7269\u5b66\u7814\u7a76\u793e\u533a\u4f7f\u7528\u3002\u901a\u8fc7\u6574\u5408\u5230 OpenFold \u4e2d\uff0c\u8fd9\u4e9b\u5b9a\u5236\u5185\u6838\u5728\u8bad\u7ec3\u671f\u95f4\u63d0\u4f9b\u4e86\u663e\u8457\u7684\u52a0\u901f\uff0c\u5e76\u663e\u8457\u51cf\u5c11\u4e86\u6a21\u578b\u7684\u8bad\u7ec3\u548c\u63a8\u7406\u7684\u5cf0\u503c\u5185\u5b58\u9700\u6c42\u3002\u8fd9\u4f7f\u5f97 OpenFold \u53ef\u4ee5\u7528\u66f4\u5927\u3001\u66f4\u590d\u6742\u7684\u6a21\u578b\uff0c\u4f7f\u7528\u66f4\u957f\u7684\u5e8f\u5217\u5728\u66f4\u5e7f\u6cdb\u7684\u786c\u4ef6\u4e0a\u8fdb\u884c\u5b9e\u9a8c\u3002\u5173\u4e8e\u8fd9\u9879\u6280\u672f\u7684\u8be6\u7ec6\u4fe1\u606f\u53ef\u4ee5\u5728\u8fd9\u91cc\u627e\u5230\uff1ahttps:\/\/deepspeed4science.ai\/2023\/09\/18\/model-showcase-openfold\/<\/p>\n\n\n\n

          \u5c55\u793a\uff08II\uff09\uff1aDeepSpeed4Science \u901a\u8fc7\u7cfb\u7edf\u548c\u7b97\u6cd5\u65b9\u6cd5\u4e3a\u57fa\u56e0\u7ec4\u57fa\u7840\u6a21\u578b\uff08\u4f8b\u5982\uff0cGenSLMs\uff09\u63d0\u4f9b\u957f\u5e8f\u5217\u652f\u6301<\/strong><\/p>\n\n\n\n

          <\/p>\n\n\n\n

          \"\u56fe8\uff1aGenSLMs\uff1a\u83b72022\u5e74
          \u56fe8\uff1aGenSLMs\uff1a\u83b72022\u5e74 ACM \u6208\u767b\u8d1d\u5c14\u5956\u7684 COVID \u57fa\u56e0\u7ec4\u6a21\u578b\uff08\u57fa\u4e8e GPT-NeoX \u7684 25B\/33B \u6a21\u578b\uff09\u3002\u5b83\u7528\u4e8e\u5b66\u4e60\u63cf\u8ff0 SARS-CoV-2 \u57fa\u56e0\u7ec4\u751f\u7269\u5b66\u610f\u4e49\u7684\u6f5c\u5728\u7a7a\u95f4\u3002\u8fd9\u4e2a GIF \u5c55\u793a\u4e86\u4e00\u4e2a\u91cd\u8981\u7684\u86cb\u767d\u8d28\u5bb6\u65cf\u82f9\u679c\u9178\u8131\u6c22\u9176\uff08malate dehydrogenase\uff09\u7684\u6839\u636e\u91cd\u8981\u7279\u5f81\u7740\u8272\u7684\u6f5c\u5728\u7a7a\u95f4\u7684\u6295\u5f71\u3002<\/figcaption><\/figure>\n\n\n\n

          GenSLMs\uff0c\u4e00\u4e2a\u6765\u81ea\u963f\u8d21\u56fd\u5bb6\u5b9e\u9a8c\u5ba4\u76842022\u5e74 ACM \u6208\u767b\u8d1d\u5c14\u5956\u83b7\u5956\u7684\u57fa\u56e0\u7ec4\u6a21\u578b\uff0c\u53ef\u4ee5\u901a\u8fc7\u5927\u578b\u8bed\u8a00\u6a21\u578b\uff08LLMs\uff09\u7684\u57fa\u56e0\u7ec4\u6570\u636e\u8bad\u7ec3\u6765\u5b66\u4e60 SARS-CoV-2\uff08COVID-19\uff09\u57fa\u56e0\u7ec4\u7684\u8fdb\u5316\u3002\u5b83\u65e8\u5728\u6539\u53d8\u5982\u4f55\u8bc6\u522b\u548c\u5206\u7c7b\u5f15\u53d1\u5927\u6d41\u884c\u7684\u75c5\u6bd2\uff08\u7279\u522b\u662f SARS-CoV-2\uff09\u7684\u65b0\u53d8\u79cd\u3002GenSLMs \u4ee3\u8868\u4e86\u7b2c\u4e00\u6279\u53ef\u4ee5\u6cdb\u5316\u5230\u5176\u4ed6\u9884\u6d4b\u4efb\u52a1\u7684\u57fa\u56e0\u7ec4\u57fa\u7840\u6a21\u578b\u3002\u5bf9\u6f5c\u5728\u7a7a\u95f4\u7684\u826f\u597d\u7406\u89e3\u53ef\u4ee5\u5e2e\u52a9 GenSLMs \u5904\u7406\u8d85\u51fa\u4ec5\u4ec5\u662f\u75c5\u6bd2\u5e8f\u5217\u7684\u65b0\u9886\u57df\uff0c\u5e76\u6269\u5c55\u5b83\u4eec\u6a21\u62df\u7ec6\u83cc\u75c5\u539f\u4f53\u751a\u81f3\u771f\u6838\u751f\u7269\u7684\u80fd\u529b\uff08\u4f8b\u5982\uff0c\u7406\u89e3\u529f\u80fd\u3001\u9014\u5f84\u6210\u5458\u8d44\u683c\u548c\u8fdb\u5316\u5173\u7cfb\u7b49\u4e8b\u7269\uff09\u3002\u4e3a\u4e86\u5b9e\u73b0\u8fd9\u4e00\u79d1\u5b66\u76ee\u6807\uff0cGenSLMs \u548c\u7c7b\u4f3c\u7684\u6a21\u578b\u9700\u8981\u975e\u5e38\u957f\u7684\u5e8f\u5217\u652f\u6301\u7528\u4e8e\u8bad\u7ec3\u548c\u63a8\u7406\uff0c\u8fd9\u8d85\u51fa\u4e86\u50cf FlashAttention \u8fd9\u6837\u7684\u901a\u7528 LLM \u7684\u957f\u5e8f\u5217\u7b56\u7565\u3002\u901a\u8fc7 DeepSpeed4Science \u7684\u65b0\u8bbe\u8ba1\uff0c\u79d1\u5b66\u5bb6\u73b0\u5728\u53ef\u4ee5\u6784\u5efa\u548c\u8bad\u7ec3\u5177\u6709\u663e\u8457\u66f4\u957f\u7684\u4e0a\u4e0b\u6587\u7a97\u53e3\u7684\u6a21\u578b\uff0c\u5141\u8bb8\u4ed6\u4eec\u63a2\u7d22\u4ee5\u524d\u65e0\u6cd5\u8bbf\u95ee\u7684\u5173\u7cfb\u3002<\/p>\n\n\n\n

          <\/p>\n\n\n\n

          \"\u56fe9\uff1a\u7531\u4e0d\u540c\u6846\u67b6\u5728\u4e0d\u540c\u89c4\u6a21\u4e0b\u652f\u6301\u7684\u4e24\u4e2a
          \u56fe9\uff1a\u7531\u4e0d\u540c\u6846\u67b6\u5728\u4e0d\u540c\u89c4\u6a21\u4e0b\u652f\u6301\u7684\u4e24\u4e2a GenSLMs \u6a21\u578b\u7684\u6700\u5927\u5e8f\u5217\u957f\u5ea6\u3002\u4f7f\u7528 NVIDIA DGX\uff0c\u6bcf\u4e2a\u8282\u70b9\u6709\u516b\u4e2a40G A100 GPU\u3002<\/figcaption><\/figure>\n\n\n\n

          \u5177\u4f53\u5728\u7cfb\u7edf\u5c42\u9762\uff0c\u6211\u4eec\u53d1\u5e03\u4e86\u5305\u62ec\u957f\u5e8f\u5217\u652f\u6301\u548c\u5176\u4ed6\u65b0\u4f18\u5316\u7684\u6700\u65b0\u7684 Megatron-DeepSpeed \u6846\u67b6\u3002\u79d1\u5b66\u5bb6\u73b0\u5728\u53ef\u4ee5\u901a\u8fc7\u6211\u4eec\u65b0\u6dfb\u52a0\u7684\u5185\u5b58\u4f18\u5316\u6280\u672f\uff08\u5982\u6ce8\u610f\u529b\u63a9\u7801\u5f02\u6b65\u5904\u7406\u548c\u4f4d\u7f6e\u7801\u5206\u5272\uff09\u3001\u5f20\u91cf\u5e76\u884c\u3001\u6d41\u6c34\u7ebf\u5e76\u884c\u3001\u5e8f\u5217\u5e76\u884c\u3001\u57fa\u4e8e ZeRO \u7684\u6570\u636e\u5e76\u884c\u548c\u6a21\u578b\u72b6\u6001\u5f02\u6b65\u5904\u7406\u7b49\u6280\u672f\u7684\u534f\u540c\u7ec4\u5408\uff0c\u7528\u66f4\u957f\u7684\u5e8f\u5217\u8bad\u7ec3\u4ed6\u4eec\u7684 GenSLMs \u7b49\u5927\u578b\u79d1\u5b66\u6a21\u578b\u3002\u56fe9\u5c55\u793a\u4e86\u6211\u4eec\u7684\u65b0\u7248\u672c\u4f7f GenSLMs \u768425B\u548c33B\u6a21\u578b\u7684\u6700\u957f\u5e8f\u5217\u957f\u5ea6\u5206\u522b\u6bd4\u4e4b\u524d\u7684 Megatron-DeepSpeed \u7248\u672c\u589e\u52a0\u4e8612\u500d\u548c14\u500d\u3002\u5728\u652f\u6301\u7684\u5e8f\u5217\u957f\u5ea6\u65b9\u9762\uff0c\u8fd9\u4e2a\u65b0 Megatron-DeepSpeed \u6846\u67b6\u4e5f\u663e\u8457\u5730\u8d85\u8fc7\u4e86 NVIDIA \u7684 Megatron-LM\uff08\u5bf9\u4e8e25B\u548c33B\u6a21\u578b\u5206\u522b\u9ad8\u8fbe9.8\u500d\u548c9.1\u500d\uff09\u3002\u4f8b\u5982\uff0c\u963f\u8d21\u5b9e\u9a8c\u5ba4\u56e2\u961f\u7684 GenSLMs 25B \u6a21\u578b\u572864\u4e2a GPU \u4e0a\u7684\u539f\u59cb\u5e8f\u5217\u957f\u5ea6\u4e3a42K\uff0c\u800c\u73b0\u5728\u53ef\u4ee5\u7528512K\u7684\u6838\u82f7\u9178\u5e8f\u5217\u8fdb\u884c\u8bad\u7ec3\u3002\u8fd9\u5728\u4e0d\u635f\u5931\u51c6\u786e\u6027\u7684\u6761\u4ef6\u4e0b\u5927\u5927\u63d0\u9ad8\u4e86\u6a21\u578b\u8d28\u91cf\u548c\u79d1\u5b66\u53d1\u73b0\u7684\u8303\u56f4\u3002\u5bf9\u4e8e\u90a3\u4e9b\u66f4\u559c\u6b22\u76f8\u5bf9\u4f4d\u7f6e\u7f16\u7801\u6280\u672f\u8fd9\u6837\u7684\u7b97\u6cd5\u7b56\u7565\u7684\u9886\u57df\u79d1\u5b66\u5bb6\uff0c\u8fd9\u4e2a\u65b0\u7248\u672c\u4e5f\u8fdb\u884c\u4e86\u96c6\u6210\u3002<\/p>\n\n\n\n

          \u603b\u7ed3\u548c\u8def\u7ebf\u56fe<\/h2>\n\n\n\n

          \u6211\u4eec\u975e\u5e38\u81ea\u8c6a\u548c\u5174\u594b\u5730\u5ba3\u5e03 DeepSpeed4Science \u8ba1\u5212\u4ee5\u53ca\u51e0\u4e2a\u7814\u53d1\u4eae\u70b9\u548c\u6210\u679c\u3002\u4ece\u4eca\u5929\u5f00\u59cb\uff0c\u6211\u4eec\u5c06\u5728 deepspeed4science.ai \u4e0a\u4ecb\u7ecd\u6211\u4eec\u7684\u65b0\u8ba1\u5212\uff0c\u5305\u62ec\u5173\u4e8e\u6211\u4eec\u7684\u5916\u90e8\u5408\u4f5c\u8005\u7684\u4fe1\u606f\uff0c\u4ee5\u53ca\u5f53\u524d\u548c\u672a\u6765\u7684 DeepSpeed4Science \u6280\u672f\u53d1\u5e03\u3002\u6211\u4eec\u7684\u4e00\u4e2a\u9ad8\u5c42\u6b21\u76ee\u6807\u662f\u63a8\u5e7f\u5e7f\u6cdb\u89e3\u51b3\u5927\u89c4\u6a21\u79d1\u5b66\u53d1\u73b0\u7684\u4e3b\u8981\u7cfb\u7edf\u75db\u70b9\u7684 AI \u7cfb\u7edf\u6280\u672f\u3002\u6211\u4eec\u5e0c\u671b\u5168\u7403\u7684\u79d1\u5b66\u5bb6\u4eec\u80fd\u591f\u4ece DeepSpeed4Science \u901a\u8fc7\u5f00\u6e90\u8f6f\u4ef6\u89e3\u9501\u7684\u65b0\u529f\u80fd\u4e2d\u53d7\u76ca\u3002\u6211\u4eec\u671f\u5f85\u66f4\u597d\u5730\u4e86\u89e3\u963b\u788d\u60a8\u7684\u79d1\u5b66\u53d1\u73b0\u7684 AI \u7cfb\u7edf\u8bbe\u8ba1\u6311\u6218\u3002\u6211\u4eec\u771f\u8bda\u5730\u6b22\u8fce\u60a8\u7684\u53c2\u4e0e\uff0c\u5e2e\u52a9\u6784\u5efa\u4e00\u4e2a\u66f4\u6709\u524d\u9014\u7684 AI4Science \u672a\u6765\u3002\u8bf7\u7ed9\u6211\u4eec\u53d1\u9001\u7535\u5b50\u90ae\u4ef6\u81f3 deepspeed-info@microsoft.com\u3002\u6211\u4eec\u9f13\u52b1\u60a8\u5728\u6211\u4eec\u7684 DeepSpeed GitHub \u4e0a\u62a5\u544a\u95ee\u9898\u3001\u8d21\u732e PR\u3001\u53c2\u4e0e\u8ba8\u8bba\u3002<\/p>\n\n\n\n

          \u81f4\u8c22<\/em><\/p>\n\n\n\n

          Core DeepSpeed4Science Team:<\/em><\/strong><\/p>\n\n\n\n

          Shuaiwen Leon Song (DeepSpeed4Science lead), Minjia Zhang, Conglong Li, Shiyang Chen, Chengming Zhang, Xiaoxia (Shirley) Wu, Masahiro Tanaka, Martin Cai, Adam Graham, Charlie Zhou, Yuxiong He (DeepSpeed team lead)<\/em><\/p>\n\n\n\n

          Our Founding Collaborators (in alphabetical order):<\/em><\/strong><\/p>\n\n\n\n

          Argonne National Lab team:<\/strong> Rick Stevens, Cristina Negri, Rao Kotamarthi, Venkatram Vishwanath, Arvind Ramanathan, Sam Foreman, Kyle Hippe, Troy Arcomano, Romit Maulik, Maxim Zvyagin, Alexander Brace, Yuntian Deng, Bin Zhang, Cindy Orozco Bohorquez, Austin Clyde, Bharat Kale, Danilo Perez-Rivera, Heng Ma, Carla M. Mann, Michael Irvin, J. Gregory Pauloski, Logan Ward, Valerie Hayot, Murali Emani, Zhen Xie, Diangen Lin, Maulik Shukla, Weili Nie, Josh Romero, Christian Dallago, Arash Vahdat, Chaowei Xiao, Thomas Gibbs, Ian Foster, James J. Davis, Michael E. Papka, Thomas Brettin, Anima Anandkumar<\/em><\/p>\n\n\n\n

          AMD:<\/strong> Ivo Bolsen, Micheal Schulte, Bo Begole, Angela Dalton, Steve Reinhart, Ashwin Aji, Jalal Mahmud, Mahesh Balashibramanian<\/em><\/p>\n\n\n\n

          Brookhaven National Lab team:<\/strong> Adolfy Hoisie, Shinjae Yoo, Yihui Ren.<\/em><\/p>\n\n\n\n

          Columbia University OpenFold team:<\/strong> Mohammed AlQuraishi, Gustaf Ahdritz<\/em><\/p>\n\n\n\n

          Microsoft Research AI4Science team:<\/strong> Christopher Bishop, Bonnie Kruft, Tie-Yan Liu, Christian Bodnar, Wessel Bruinsma, Chan Cao, Yuan-Jyue Chen, Peggy Dai, Patrick Garvan, Liang He, Elizabeth Heider, PiPi Hu, Peiran Jin, Fusong Ju, Yatao Li, Chang Liu, Renqian Luo, Qi Meng, Frank No\u00e9, Tao Qin, Janwei Zhu, Bin Shao, Yu Shi, Wenlei Shi, Gregor Simm, Megan Stanley, Lixin Sun, Yue Wang, Tong Wang, Zun Wang, Lijun Wu, Yingce Xia, Leo Xia, Shufang Xie, Shuxin Zheng, Jianwei Zhu<\/em><\/p>\n\n\n\n

          Oakridge National Lab team:<\/strong> Prassana Balaprakash, Georgia Tourass<\/em><\/p>\n\n\n\n

          Princeton University:<\/strong> William Tang, Kyle Felker, Alexey Svyatkovskiy (Microsoft liaison)<\/em><\/p>\n\n\n\n

          Rutgers University:<\/strong> Hang Liu<\/em><\/p>\n\n\n\n

          WebXT Weather team:<\/strong> Pete Luferenko, Divya Kumar, Jonathan Weyn, Ruixiong Zhang, Sylwester Klocek, Volodymyr Vragov<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

          \u672c\u6587\u8f6c\u8f7d\u81ea\u5fae\u8f6f DeepSpeed \u56e2\u961f\u5b98\u65b9\u77e5\u4e4e\u8d26\u53f7\uff1azhihu.com\/people\/deepspeed\uff0c\u7531\u5fae\u8f6f DeepSpeed \u56e2\u961f\u7ffb\u8bd1\u81ea\u5b98\u65b9\u82f1\u6587\u535a\u5ba2\uff1aAnnouncing the DeepSpeed4Science Initiative: Enabling large-scale scientific discovery through sophisticated AI system technologies\uff0c\u672c\u6587\u7565\u4f5c\u8c03\u6574\u3002 \u5728\u63a5\u4e0b\u6765\u7684\u5341\u5e74\u4e2d\uff0c\u6df1\u5ea6\u5b66\u4e60\u53ef\u80fd\u4f1a\u5f7b\u5e95\u6539\u53d8\u81ea\u7136\u79d1\u5b66\uff0c\u589e\u5f3a\u6211\u4eec\u5bf9\u81ea\u7136\u73b0\u8c61\u8fdb\u884c\u5efa\u6a21\u548c\u9884\u6d4b\u7684\u80fd\u529b\u3002\u8fd9\u53ef\u80fd\u9884\u793a\u7740\u79d1\u5b66\u63a2\u7d22\u7684\u65b0\u65f6\u4ee3\uff0c\u4e3a\u4ece\u836f\u7269\u5f00\u53d1\u5230\u53ef\u518d\u751f\u80fd\u6e90\u7684\u5404\u4e2a\u9886\u57df\u5e26\u6765\u91cd\u5927\u8fdb\u5c55\u3002\u4e3a\u4e86\u54cd\u5e94\u8fd9\u4e00\u673a\u4f1a\u4ee5\u53ca\u5fae\u8f6f\u201c\u4e88\u529b\u5168\u7403\u6bcf\u4e00\u4eba\u3001\u6bcf\u4e00\u7ec4\u7ec7\uff0c\u6210\u5c31\u4e0d\u51e1\u201d\u7684\u4f7f\u547d\uff0c\u5fae\u8f6f DeepSpeed \u56e2\u961f\u542f\u52a8\u4e86\u4e00\u4e2a\u540d\u4e3a DeepSpeed4Science \u7684\u65b0\u8ba1\u5212\uff0c\u65e8\u5728\u901a\u8fc7 AI \u7cfb\u7edf\u6280\u672f\u521b\u65b0\u5e2e\u52a9\u9886\u57df\u4e13\u5bb6\u89e3\u9501\u5f53\u4eca\u6700\u5927\u7684\u79d1\u5b66\u4e4b\u8c1c\u3002 DeepSpeed \u7cfb\u7edf\u662f\u7531\u5fae\u8f6f\u5f00\u53d1\u7684\u4e1a\u754c\u9886\u5148\u7684\u5f00\u6e90 AI \u7cfb\u7edf\u6846\u67b6\uff0c\u5b83\u4e3a\u5404\u79cd AI \u786c\u4ef6\u4e0a\u7684\u6df1\u5ea6\u5b66\u4e60\u8bad\u7ec3\u548c\u63a8\u7406\u63d0\u4f9b\u4e86\u524d\u6240\u672a\u6709\u7684\u89c4\u6a21\u548c\u901f\u5ea6\u3002\u56fe1\u5c55\u793a\u4e86\u6211\u4eec\u5bf9 DeepSpeed4Science \u8fd9\u4e00\u65b0\u8ba1\u5212\u7684\u57fa\u672c\u65b9\u6cd5\u3002\u901a\u8fc7\u5229\u7528 DeepSpeed \u5f53\u524d\u7684\u6280\u672f\u65b9\u6848\uff08\u8bad\u7ec3\u3001\u63a8\u7406\u548c\u538b\u7f29\uff09\u4f5c\u4e3a\u57fa\u7840\u6280\u672f\u63a8\u52a8\u5668\uff0cDeepSpeed4Science \u5c06\u521b\u5efa\u4e00\u5957\u4e13\u4e3a\u52a0\u901f\u79d1\u5b66\u53d1\u73b0\u800c\u91cf\u8eab\u5b9a\u5236\u7684 AI \u7cfb\u7edf\u6280\u672f\uff0c\u4ee5\u5e94\u5bf9\u5176\u72ec\u7279\u7684\u590d\u6742\u6027\uff0c\u8d85\u8d8a\u7528\u4e8e\u52a0\u901f\u901a\u7528\u5927\u578b\u8bed\u8a00\u6a21\u578b\uff08LLMs\uff09\u7684\u5e38\u89c1\u6280\u672f\u65b9\u6cd5\u3002\u6211\u4eec\u4e0e\u62e5\u6709\u79d1\u5b66 AI \u6a21\u578b\u7684\u5185\u90e8\u548c\u5916\u90e8\u56e2\u961f\u7d27\u5bc6\u5408\u4f5c\uff0c\u4ee5\u53d1\u73b0\u548c\u89e3\u51b3\u9886\u57df\u7279\u5b9a AI \u7cfb\u7edf\u7684\u6311\u6218\u3002\u8fd9\u5305\u62ec\u6c14\u5019\u79d1\u5b66\u3001\u836f\u7269\u8bbe\u8ba1\u3001\u751f\u7269\u5b66\u7406\u89e3\u3001\u5206\u5b50\u52a8\u529b\u5b66\u6a21\u62df\u3001\u764c\u75c7\u8bca\u65ad\u548c\u76d1\u6d4b\u3001\u50ac\u5316\u5242\/\u6750\u6599\u53d1\u73b0\u3001\u548c\u5176\u4ed6\u9886\u57df\u3002 \u6211\u4eec\u7684\u957f\u671f\u613f\u666f\u662f\u5c06 DeepSpeed4Science \u53d1\u5c55\u6210\u4e00\u4e2a\u7528\u4e8e\u5206\u4eab\u652f\u6301\u79d1\u5b66\u53d1\u73b0\u7684\u5148\u8fdb AI \u6280\u672f\u7684\u8f6f\u4ef6\u5e73\u53f0\u548c\u7edf\u4e00\u4ee3\u7801\u4ed3\u5e93\u3002DeepSpeed4Science \u7684\u8bbe\u8ba1\u65e8\u5728\u5305\u5bb9\u6027\uff0c\u547c\u5e94\u5fae\u8f6f\u7684\u201cAI for Good\u201d\u627f\u8bfa\u3002\u8fd9\u4f53\u73b0\u5728\u8be5\u8ba1\u5212\u5bf9\u4e00\u7cfb\u5217\u6807\u5fd7\u6027\u79d1\u5b66\u6a21\u578b\u7684\u652f\u6301\u4e0a\uff0c\u4ed6\u4eec\u4ee3\u8868\u4e86\u4e00\u4e9b\u6700\u5173\u952e\u7684 AI4Science \u5e94\u7528\u573a\u666f\u3002\u5728\u8fd9\u7bc7\u535a\u5ba2\u4e2d\uff0c\u6211\u4eec\u5c55\u793a\u4e86 DeepSpeed4Science \u5982\u4f55\u5e2e\u52a9\u89e3\u51b3\u7ed3\u6784\u751f\u7269\u5b66\u7814\u7a76\u4e2d\u7684\u4e24\u4e2a\u5173\u952e AI \u7cfb\u7edf\u6311\u6218\uff1a(1) […]<\/p>\n","protected":false},"author":34512,"featured_media":1075587,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-content-parent":1012650,"msr_hide_image_in_river":0,"footnotes":""},"research-area":[],"msr-locale":[268881],"msr-post-option":[],"class_list":["post-1073181","msr-blog-post","type-msr-blog-post","status-publish","has-post-thumbnail","hentry","msr-locale-zh_cn"],"msr_assoc_parent":{"id":1012650,"type":"lab"},"_links":{"self":[{"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/1073181","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post"}],"about":[{"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-blog-post"}],"author":[{"embeddable":true,"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/users\/34512"}],"version-history":[{"count":2,"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/1073181\/revisions"}],"predecessor-version":[{"id":1075635,"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-blog-post\/1073181\/revisions\/1075635"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media\/1075587"}],"wp:attachment":[{"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1073181"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1073181"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1073181"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/new-cm-edgedigital.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1073181"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}