§@ªÌ |
¼ÐÃD ![¤U¤@Ó¼ÐÃD ¤U¤@Ó¼ÐÃD](icon_go_right.gif) |
guy123
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
2557 Posts |
Posted - 02/02/2025 : 08:00:08
|
quote: Originally posted by Henschel
¤f国ªºdeepshit¥Î^°¶¹F´¹¤ù¡A^°¶¹F·íµM»¡¦Û¤v«È¤á¬O³Ì¥ý¶iªº¤j»y¨¥¼Ò«¬![](icon_smile_cool.gif)
==== ¤f¤f¡A这¤\«ë°Ú¡ã¡ã¡ã§A狭¹i¤F§a¡Adeepseek¬O¥i¥H¤£¥Î^伟达¡A¬Æ¦Ü¤£¥ÎGPUªº¡AReddit¦³¥Î户¦¨¥\¤£¨Ï¥ÎGPU运¦æ§¹¾ãª©R1¡C
¥t¥~¡A¡§Öº°ò¬y动©M华为¤ª«Å¥¬¡G¦}¤W线°ò¤_华为¤ªª@腾¤ªªA务ªºDeepSeekR1/V3±À²zªA务¡C¡¨¥þ¦Û¥Dªºª@腾ªä¤ù¡B¤ªªA务+Ãz´ÚDeepseek¤j¼Ò«¬¡C
¦´Nª¾¹D¦Ñ¬ü会¥Ç贱¨Ï§¥¤F¡A¯dþÓ¦Z¤â©O¡C¤f岛§A还¬O·Q·Q¦Û¨¦p¦ó¹³§A¬ü¯R¥æ税§a¡C
https://weibo.com/2868676035/5129320169211617
Telegram 创©l¤H¤]¦b¤ÀªRdeepseekªº¦¨¥\¦Z¡A²r¦j¤¤国ªº±Ð¨|Ê^¨t¡A
🔹¡§随þÓ¤¤国ªì创¤½¥q DeepSeek ªº¦¨¥\¡A许¦h¤H对¤¤国¦b¤H¤u´¼¯à领°ìÓã¤W¬ü国ªº³t«×·P¨ìÕa讶¡CµM¦Ó¡A¤¤国¦bºâªk®Ä²v¤è±ªº进¨B¦}«Dú«ªÅ¦Ó来¡C¤¤国学¥Í¦b国际奥ªL¤Ç§J竞赛¤¤数学©M编µ{¤è±ªºªí现¦¤w¶W¶V¨ä¥L学¥Í¡C¡¨ 🔹¡§¦b°ö养数学©M¬ì学领°ìªºÉ¬¨q¤H¤~¤è±¡A¤¤国ªº¤¤µ¥±Ð¨|Ê^¨tɬ¤_¦è¤è¡C¥¦¹ª励学¥Í¤§间¿E¯P竞争¡A¬Û¤ñ¤§¤U¡A¤j¦h数¦è¤è学®Õ¤£¹ª励竞争¡A¸T¤î¤½开¤½¥¬学¥Íªº¦¨绩©M±Æ¦W¡C
https://weibo.com/1671109627/5128521106069408
¡§Reddit ¥Î户¦¨¥\¤£¨Ï¥ÎGPU运¦æ§¹¾ãª©R1
¥L³q过¸T¥Î3090TI显¥d¡A¦b96GB内¦s¥DÉó¤W¨Ï¥Îllama.cppªºmmap§Þ术¡A将¼Ò«¬权«¦s储¦bNVMe©T态µw盘¤¤¡A§Q¥Î¨t统内¦s§@为ºÏ盘缓¦s¡A实现约2.13 token/¬íªº±À²z³t«×¡C²~颈¦b¤_PCIe 5.0 x4©T态ªº读¨ú³t«×¡C¡¨ https://weibo.com/2672926347/5129234213243964 |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
Henschel
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
3386 Posts |
Posted - 02/02/2025 : 11:36:01
|
deepshitªº¾Ç²ß¼Ò«¬ªÖ©w¤£¥ÎGPU
¤£ª¾"¥þ¦Û¥D"ªºª@腾¶]¤£¶]±o°Ê¤f国¸£´Ý¤W¤@Ó¸¾÷ªM¶Â®©ªÅ
¤f国¿Å¤ô¤¤¾Çªº¦Û±þ²v©M¾¹©x®½Ãؤñ²v·íµMÀu©ó¦è¤è
|
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
yangly3
§Ú¬O¦Ñ³¾
![](icon_star_purple.gif) ![](icon_star_purple.gif) ![](icon_star_purple.gif) ![](icon_star_purple.gif)
7939 Posts |
Posted - 02/02/2025 : 13:02:30
|
quote: Originally posted by guy123
[quote]Originally posted by Henschel
随þÓ¤¤国ªì创¤½¥q DeepSeek ªº¦¨¥\¡A许¦h¤H对¤¤国¦b¤H¤u´¼¯à领°ìÓã¤W¬ü国ªº³t«×·P¨ìÕa讶
¤p¯Ý®X¤X¡A¤D说¤¤国·d±o这¨Ç东¦è¡A¥¦们ªº´¼¤O¯à¤£¯à¶W过习¥D®u鸭 --- ¤D¯à给个ÚÌ数¤I¤\¡H
![](https://i.postimg.cc/Qd7xQNMw/470544869-1145873833559786-3542031295183694521-n.jpg) |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
guy123
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
2557 Posts |
Posted - 02/02/2025 : 13:07:52
|
[quote]Originally posted by Henschel
deepshitªº¾Ç²ß¼Ò«¬ªÖ©w¤£¥ÎGPU
¤£ª¾"¥þ¦Û¥D"ªºª@腾¶]¤£¶]±o°Ê¤f国¸£´Ý¤W¤@Ó¸¾÷ªM¶Â®©ªÅ
¤f国¿Å¤ô¤¤¾Çªº¦Û±þ²v©M¾¹©x®½Ãؤñ²v·íµMÀu©ó¦è¤è
==== ÆÓª¾¤f¤fÔ£³£¤£À´还¦b这卖§Ë¡A连华为ª@腾¥Dn¬O卖给¥ø业¥Î¤_ªA务¾¹³£¤£ª¾¹D¡ã¥þ国¦U¤j´¼ºâ¤¤¤ß°ò¥»³£¬O¥Hª@腾为¥D¤F¡C
¤f¤f¥Î¤WÉE¬Æ¦Ü¤LÉE¬üª÷ªºGPuªä¤ù¶]¶Â®©ªÅ¡A¤@¦V³£这¤\凯¤lªº¹À¡H ¿Å¤ô¤¤学ªº¦Û杀²vÉO§Ú们讨论ªº¥D题¦³¤°¤\关¨t¡Hdeepseekªº®Ö¤ß¦¨员´N¦³¿Å¤ô¤¤学毕业ªº¡A¿Å¤ô¤¤学¦Û杀²v¤]没¦³¬ü«Ò°ª¡A¬ü«Ò还¦³®Õû\枪杀¡ã¡ã¡ã辩¤£过´N开©l½ö¦aA赖转²¾话题¡A©M§A¬ü¯R®¼¹³ªº¡A竞争¤£过´N开©lºN¶Â«Ê杀¤U¤T滥¡C
¥t¥~¬ü«Ò¥|¤Ñ±¼¥|¬[飞Éó¡C«Ò国«ç¤\¦¨这个ý©¤l¤F¡ã¡ã¡ã |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
Henschel
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
3386 Posts |
Posted - 02/02/2025 : 14:25:02
|
¤f国¸£´Ý¦Û¤v»¡: "¤£¨Ï¥ÎGPU¹B¦æ§¹¾ãª©R1" µ²ªG¤H®a¤@»¡¾Ç²ß¼Ò«¬´NÂàÀY§j¤f国GPU
¤f国¸£´Ý§j¤f国GPU¥Î©ó¥ø·~ªA°È¾¹¡A¨º§ó²³æªº®a¥Î¹CÀ¸¤£¬O¤â¨ì®³¨Ó¶Ü![](icon_smile_tongue.gif)
¤f国¸£´Ý§j¤f国¤¤µ¥±Ð¨|¡A«ç»ò¥i¥H¤£´£¤f国¤¤µ¥±Ð¨|¨å«¬¿Å¤ô¤¤¾Ç ÁÙ¦³¤f国¤¤µ¥±Ð¨|»»»»»â¥ýªº¾¹©x²£¥X²v![](icon_smile_tongue.gif)
¹ï¤F¡A¬ü°êµØ²±¹y±¼ªº«È¾÷¤WªºµØ¸Ç·Æ¦B¿ï¤â https://www.voachinese.com/a/dc-plane-crash-edward-zhou-20240131/7958913.html ¤f国¸£´Ý¥s¦b¬ü°êªº¤f国¹´»â©ñ·Ïªá§U¼y¤F¨S¦³ (¦P§Óªº«Øij ) |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
Light
§Ú¬Oµæ³¾
![](icon_star_orange.gif) ![](icon_star_orange.gif)
503 Posts |
Posted - 02/02/2025 : 14:29:59
|
Deepseek ¬O¾ã¦X²{¤µAI§Þ³N¡A¥[¤p¨B³Ð·s¡A¤£¬O¤°»ò°ò¦¬ð¯}¡F³o¬O²{¤µAIµo®iªº¼Ò¦¡¡A¯¸¦b¥L¤HªÓ»H¤W¿N¿ú¡A¨S¤H¥i§K¡C³Ìªñ³Q§j±o³o»ò¼ö¡A®£¦³¤º¥~«Åªº±À¤â¡C
²Ó¬d¹êÃÒ¡C
1. ¬ð¯}©Ê¥u»Ý¤Q¤À¤§¤@ºâ¤O¡G§j¡C
2023¦~«e¡ALLM¥D¬y¬O dense ¼Ò¦¡¡A«Ü¯Óºâ¤O¡C2023¦~2¤ë¡AGPT-4§ï¥Î MoE ¬Ùºâ¤O¡A2024¦~3¤ë¡AGoogle Gemini 1.5 ¸ò¶i¤]¥Î MoE¡CµM«á2024¦~¥X¤F¤@°ï¶}·½ MoE¡A¥]¬A Deepseek¡C
MoE ¤ñ dense ªºÀu¶Õ¡A´N¦b©ó Na ¤p¡A¥i¬O¨C¤@®a MoE ³£¦³¦P¼ËªºÀu¶Õ¡C©Ò¥H Deepseek ¥u®³ Meta ©M Alibaba ªº dense ¨Ó¤ñ¡A¥i¨S´±»¡¤ñ¨ä¥¦ MoE ¬Ùºâ¤O¡C
![](https://i.imgur.com/mizIKsO.jpeg)
2. Deepseek ¶}µo§C¦¨¥»¡G»~¸Ñ¡Ï¥~«Å§j
AI ´N¬O¿N¿ú¡C±ç¤å¾W¦b³Ð¿ì deepseek ®É»¨®ðºÙ¤w¶R¤F¤@¸U¤ùA100¡C
Deepseek V3 »¡ªº 560¸U¬ü¤¸¡A¬O¶]¤@¦¸¥¿¦¡°V½m¯Ó¥ÎH800 GPU ®É¼Æ x $2/¤p®Éªº¯Â¾÷¾¹¶O¥Î¡C¤§«e¬ã¨s¡Bµo®i¼Ò«¬¡B»`¶°°V½m¸ê®Æ¡B¸Õ¶]¡B½Õ¸Õ¡B¥¢±Ñ«¶]µ¥µ¥¶O¥Î³£¤£¦b¤º¡C¤£»¡§Oªº¡A¶]³o55¤Ñ¡AºÞ±±¡BÀˬd¡B½Õ´ú¡BºÊµø¡B×´_ªº¤H¤u³£¤£¥Îªá¿ú°Ú¡H
Meta ªº LLaMA2 70B ¦b2023¦~¶]¤@¦¸¥¿¦¡°V½m¯Ó¥ÎA100 GPU ¦@1.72M¤p®É x $1.5/¤p®É¡A¨º¬O 258¸U¬ü¤¸¡A«ç»ò¨S¤H¸õ¥X¨Ó»¡¦n´Î´Î§C¦¨¥»¡H
DeepSeek-V3 costs only 2.788M GPU hours for its full training. Assuming the rental price of the H800 GPU is $2 per GPU hour, our total training costs amount to only $5.576M. Note that the aforementioned costs include only the official training of DeepSeek-V3, excluding the costs associated with prior research and ablation experiments on architectures, algorithms, or data.
Our main contribution includes:
3. PTX ¶¹L CUDA ³Ð·s¼W¥[ºâ¤O¡G»~¸Ñ + ¥~«Å§j
deepseek ¥Î PTX ¼g³q°Tµ{¦¡¬OµL©`¡A¬°¤F¶¹L H800 NVLink ÀW¼e¶È 400 GB/s ªº¨î¡C¥ĻâGPU¸Ì132ÓSM¸Ì¡A¼·¥X20ÓSM±Mªù¶]PTX¨Ó³B²z³q°T¡C³o¥ß°¨·l¥¢15¢HªºµwÅéºâ¤O¡Cdeepseek¦Û¤v§Þ³N³ø§i¤]µL©`¦a»¡³o¤èªk«D±`¨S¦³®Ä²v¡C
¥i¥H¬Ý¥L̪º TR https://arxiv.org/html/2412.19437v1
In DeepSeek-V3, we implement the overlap between computation and communication to hide the communication latency during computation. This significantly reduces the dependency on communication bandwidth compared to serial computation and communication. However, the current communication implementation relies on expensive SMs (e.g., we allocate 20 out of the 132 SMs available in the H800 GPU for this purpose), which will limit the computational throughput. Moreover, using SMs for communication results in significant inefficiencies, as tensor cores remain entirely -utilized.
4. ¶]¤ÀÀu³Ó¡G¤Ñ¤Ñ§ó·s¡Aťť´N¦n¡C
¬Q¤Ñªü¨½¤Ú¤Ú°¨¤W¸õ¥X¨Ó»¡¥L̪º·s Qwen2.5-Max MoE ¥þ±À£¨î Deepseek¡C©Ò¥H¶]¤À³oºØ¨Æ¡Aťť´N¦n¡A¹L¨â¤Ñ´N·|¦³¤H»¡Ä¹¹Lªü¨½¤Ú¤Ú¡C
5. NVidia §¹³J¡A§C¶¥ GPU ´N¥i½m¦¨ AI¡G°§è
H800 ºâ¤O©M H100 ¤@¼Ë¡A®t§O¦b©ó NVLink BW ³Q¬å¢w¢w 400 GB/s vs 900 GB/s¡C
¤µ¦~§ï¦¨ MoE ªº·|§ó¦h¡A½Ö´±¤£¶R§ó¦n§ó·sªº GPU¡H
¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w ¹ï¾ú¥v¬ã¨sªÌ¦Ó¨¥¡A³Ì¨ë¿EªÌ¡AµL¹L©ó¦b¾ú¥v©M¶Ç»¡¤§¶¡¡A§ä¨ì¤F³Ì§¹¾ãªº¬G¨Æ¡C¢w¢w ºq¼w |
Edited by - Light on 02/02/2025 14:39:11 |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
pcgamer
§Ú¬O¦Ñ³¾
![](icon_star_purple.gif) ![](icon_star_purple.gif) ![](icon_star_purple.gif) ![](icon_star_purple.gif)
10224 Posts |
|
guy123
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
2557 Posts |
Posted - 02/02/2025 : 20:53:40
|
¤f国¸£´Ý§j¤f国GPU¥Î©ó¥ø·~ªA°È¾¹¡A¨º§ó²³æªº®a¥Î¹CÀ¸¤£¬O¤â¨ì®³¨Ó¶Ü === ®a¥Î´å戏ªä¤ùȤL个钱°Ú¡A没¦³«õ矿¡A 没¦³AI·§©Àª£§@¤§«e¡A^伟达ªÑɲ¤~¦h¤Ö¡C
¤f国¸£´Ý§j¤f国¤¤µ¥±Ð¨|¡A«ç»ò¥i¥H¤£´£¤f国¤¤µ¥±Ð¨|¨å«¬¿Å¤ô¤¤¾Ç ÁÙ¦³¤f国¤¤µ¥±Ð¨|»»»»»â¥ýªº¾¹©x²£¥X === 别§è²H¤F¡A¤¤国³ÌüLªº´N¬O¤¤¤p学±Ð¨|¡AÆÓ¤ñ¤ã实ªº°ò础¡C
¾¹©x产¥Xªº链±µ©O¡H¤S¬O从þ¨½¥¶来ªº粪¡A¤f¤f¯u¤£·\¬O«Ë壳¦§ëLªº³Ì¨Î¥¶粪¤u¤H¡C
¦Ñ¬ü4%¤H¤f¨Ñ应¤F¥þ²y70%ªº¦å²G¬O¨Æ实§a¡H¦å浆¥X¤f¥i¬O¦Ñ¬üªº¤j©v¥Í·N©O¡C§A¥H为¦Ñ¬ü会©ñ过¾¹©x¡H
¦Ñ¬ü³Ì¾Õ长ªº¨Æ±¡´N¬O§â¦Û¤v°µ过ªº¨Æ±¡¥Î来¦Ã½°别¤H¡AµM¦Z¤f¤f这Ïú©¾¤ü´N随þÓª¯ï开©lªíºt¤F¡C
#¬ü国¤@¤j学³QÃz¤Á¤À¥X°â°h¥î¦Ñ§L遗Ê^#¡i#¬ü国¤@¤j学³QÃz长´Á¥X°âÆÓ¥D遗Ê^# https://weibo.com/1974576991/5126496526532965
|
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
guy123
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
2557 Posts |
Posted - 02/02/2025 : 20:55:55
|
quote: Originally posted by pcgamer
¡y¼g¤@Ópythonµ{¦¡¡A®i¥Ü¤@Ó²y¦b±ÛÂ઺¤»Ãä§Î¤º¼u¸õ¡A ²yÀ³¸Ó¨ü¨ì«¤O©M¼¯À¿¤Oªº¼vÅT¡A¨Ã¥B¥²¶·¹G¯u¦a±q±ÛÂ઺Àð¾À¤W¼u°_¡z https://x.com/flavioAd/status/1885449107436679394
þ¤@ÓAIªí²{ªº¤ñ¸û¯u¹ê©O¡H![](icon_smile_cool.gif) ![](icon_smile_cool.gif) ![](icon_smile_cool.gif)
=== 来点难«×°ª点ªº¡C°ª¤U¥ß§P¡G
🔻题¥Ø¡G¡§编写¤@个 python 脚¥»¡A¨C 5 ¬í钟¦b¤@个¥¿¤è§Î内¥X现¤@个¤£¦P颜¦âªº·s弹¸õ²y¡AÚÌ«O¥¿ÚÌ处²z¸I¼²检测¡A¨Ï¥¿¤è§Î缓ºC±Û转¡C¡¨ 🔻O3-mini-high (¥ª) ÉO deepseek-r1 (¥k) 对¤ñ 🔻deepseek²z¸Ñ¤û顿²Ä¤T©w«ß¡A§Y¨C¤@个§@¥Î¤O³£会产¥Í¬Ûµ¥©M¬Û¤Ïªº¤Ï§@¥Î¤O¡C https://weibo.com/1671109627/5129329699457515 |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
Henschel
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
3386 Posts |
Posted - 02/02/2025 : 22:22:33
|
¨º¤f国¸£´Ý®a¸Ì¹q¸£ªºÅã¥Ü´¹¤ù¬O¨Ó¦Û![](icon_smile_tongue.gif)
¬ü°ê¤¤µ¥±Ð¨|¨º¤Î±o¤W¤f国¤¤µ¥±Ð¨|ÅéÀ˼ÉÀŤõ¤Æ¤@±øÀs²£·~Ãì![](icon_smile_cool.gif) |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
guy123
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
2557 Posts |
Posted - 02/02/2025 : 23:32:08
|
[quote]Originally posted by Henschel
¨º¤f国¸£´Ý®a¸Ì¹q¸£ªºÅã¥Ü´¹¤ù¬O¨Ó¦Û![](icon_smile_tongue.gif)
¬ü°ê¤¤µ¥±Ð¨|¨º¤Î±o¤W¤f国¤¤µ¥±Ð¨|ÅéÀ˼ÉÀŤõ¤Æ¤@±øÀs²£·~Ãì![](icon_smile_cool.gif) ==== ¤t¬Ó¡G¡§ª¯¥£¤~¡A§Ú°_来¡A¬ü国总统¦ì¸m给§A§¤¡C¡¨¤t¬Ó刚§å评§¹¬ü国±Ð¨|¡A¤f¤f´N来顶¼²¤F¡C
¥t¥~¤f¤f§A¯à补习¤Uªº国语¹À¡H©ÎªÌ¬O¦]为§Aªº脑¬}¤Ó¤j¡A«ä维¤Ó¸õ脱¡A¤S©ÎªÌ³ß欢¥¶¨Ç²ö¦W¨ä§®ªº粪¡A让¥¿±`¤H¸ò¤£¤W§Aªº«ä¸ô¡H
当¦a时间1¤ë29¤é¡A¬ü国总统¯S®Ô´¶¦b¤@项¦æ¬F©R¥O¤¤´£¤Î¡A¡§70%ªº¤K¦~级¤¤学¥Í阅读¯à¤O®t¡A72%ªº¤K¦~级¤¤学¥Í数学¯à¤O®t¡¨¡C¯S®Ô´¶还ªí¥Ü¡A¬ü国±Ð¨|Ê^¨t没¦³达¨ì预´Á¥Ø标¡A这¥¿¦b损®`¬ü国ªº竞争¤O¡C https://weibo.com/1663072851/5128922509609047 |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
yangly3
§Ú¬O¦Ñ³¾
![](icon_star_purple.gif) ![](icon_star_purple.gif) ![](icon_star_purple.gif) ![](icon_star_purple.gif)
7939 Posts |
Posted - 02/03/2025 : 15:18:52
|
¤p¯Ý®X鸭¡Aªí¾ã»¶¨Ç¤ª¨½雾¨½ªº«£ --- 来点¤I实¦bªº罢![](icon_smile_tongue.gif)
¤p¯Ý®X鸭¡A¤D¦³¦Ì¦³问过DS»¶锅àD¤Î灵»î²`处ªº¥@纪¤§问¡G
¬D两¦Ê¤çü®¤l¨«¤Q¨½¤s¸ô¨ì©³n¤£n换ªÓªº呐 ---请务¥²务¥²n说©ú¡G¤£¬O驴¤lªº哟
==========================================================================
这个问题n¬Oµª¤£¦n¡A·S±oùÕùÕ¤£开´Ë...
¨ì时ÔÉE¤@¥¦¤@个驴µÊÉa¤@¤W来¡A¤@JO´N§â这个¤H¤u´¼¯à给¿å¦¨¤F¤H¤u´¼»Ù ©È¤]¬O¤£¤@©wªº呐 |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
waffe
§Ú¬O¦Ñ³¾
![](icon_star_purple.gif) ![](icon_star_purple.gif) ![](icon_star_purple.gif) ![](icon_star_purple.gif)
7283 Posts |
|
a2979
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
3222 Posts |
|
cph0516
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
5671 Posts |
|
guy123
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
2557 Posts |
Posted - 02/04/2025 : 08:27:03
|
°ê¦h¬ÙÃzµo¸V¬y·P©x¤èÁô°Î¡I
¯fÂûÀnÃZ¬y¤J¥«³õ ¦h¦a¾i´Þ¤á¯É¯ÉÃz®Æ¡A¬Ì±¡³sÀôÃz¡B¾i´Þ³õ·l¥¢ºG«¡F ==== 脑残¤f¤f¡A¤é¥»¬y·P¤H数ªñ¤dÉE¡A¬[¤£¦í¤f岛岛¥Á±ÆþÓ队¥hºë¯«¥À国´ÂÉo¡A这¤£¤jS³£¬y·P¯f¦º¦b¤é¥»¤F¡C¤f¤f§A¬O¤£¬O¤µ¤Ñ¤~从¬}¨½±Æ¥X来°Ú¡H说¤j陆¸V¬y·P厉®`¡A§Ú们«ç¤\没¤@点·P觉¡C
¤£¦p谈谈§A¤é妈¨º边ªº¬y·P¸vh±¡úG§a¡A¦A谈谈¤é¥»©Ô¯ãªº医疗¨t统¡A©å¦Hªº医疗§Þ术¡A还¦³§â¯f¤H当¥Ö²y来¦^½ðªºªA务ºë¯«¡C |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
guy123
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
2557 Posts |
Posted - 02/04/2025 : 08:34:34
|
²`¦`¥_²z²ö´µ¬ì¤j学±À¥X¤F¤@项«nªººâªk¡A该ºâªk将^伟达GPUªº©Ê¯à´£°ª¤F800¿¡C¥¦还¤¹许§C¦¨¥»ªºGPU处²z°ª©Ê¯à¥ô务¡C¤¤国¬ã¨s¤H员开发¤F¤@Ïú°ª©Ê¯àºâªk¡A¥i¥H¸Ñú¨®ø费ªÌGPU¤WÎ`杂ªº§÷®Æ设计问题¡AÉO传统¤èªk¬Û¤ñ¡A³t«×´£°ª¤F800¿¡C 难©Ç^伟达ªÑ²¼¤S¶^¤F¡C153块¬üªÑ¶^Ë现¦b116¡A¸õ楼ɲ¤F¡Adeepseekªº¦ÑªO¬O°µ¶q¤Æªº¡A©È¬O¦¦°µªÅ^伟达¡A赚双¥÷钱¤F¡ã¡ã¡ã
¥H¤U为Éó½¡G ¡§¥Ñ²`¦`MSU-BIT¤j学ªº¤@个¬ã¨s团队开发¡A¥Ñ²ö´µ¬ì国¥ß罗»X诺¯Á¤Ò¤j学©M¥_¨Ê²z¤u¤j学¦@¦P创¥ß¡A·sºâªk´£°ª¤F©P动¤O学¡]PD¡^ªº计ºâ®Ä²v¡APD¬O¤@Ïú«eªuªº«D§½³¡²z论¡A¥i¥H¸Ñú¨Î`杂ªºª«²z问题µõ缝¡B损§¥©M断µõµ¥问题¡C ¥¦为¸Ñú¨¦UÏú¦æ业¡]¥]¬A¯èªÅ¯è¤Ñ©M军¨Æ应¥Î¡^ªºÎ`杂Éó±ñ问题开¹@¤F·sªº¥i¯à©Ê¡A这¨Ç¦æ业ÆΪx¨Ï¥Îªºªä¤ù¦¨¥»§C·G¥B¤£¨ü¬ü国¨îµô¡C¡¨
Chinese algorithm boosts Nvidia GPU performance 800-fold in science computing
A high-performance algorithm that could solve complicated material design problems on consumer GPUs has been developed by Chinese researchers, achieving a groundbreaking 800-fold increase in speed over traditional methods. Developed by a research team at Shenzhen MSU-BIT University, co-founded by Lomonosov Moscow State University and Beijing Institute of Technology, the new algorithm enhances the computational efficiency of peridynamics (PD), a cutting edge, non-local theory that solves difficult physical issues such as cracks, damage and fractures. It opens up new possibilities for solving complex mechanical problems across various industries, including aerospace and military applications, on widely available chips that are low-cost and not subject to US sanctions. https://www.scmp.com/news/china/science/article/3296135/chinese-algorithm-boosts-nvidia-gpu-performance-800-fold-science-computing |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
Henschel
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
3386 Posts |
Posted - 02/04/2025 : 09:13:48
|
«Xù´µ¦Ñ¤jôª½±¡¦³³ø¾É»¡¤f国deepshit¬O°ò©óĬÁp§Þ³N¶}µo¡A¤f国¸£´Ý«H¤£«H?![](icon_smile_big.gif)
¤f国¸£´Ý¬JµMÂà¶iÂåÀø¡A¨º¤£¦p»¡»¡¤f国³Ì·s国®a±ÄÁʪº³Â¾KÃÄ¥i¥HÅý¯f¤H¦b¤â³N¤¤¿ô¨Óªº³ÐÁ|
![](icon_smile_cool.gif) |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
cph0516
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
5671 Posts |
Posted - 02/04/2025 : 09:21:51
|
ÅýGPU©Ê¯à´£°ª800¿ ¨ºNVªÑ»ùÀ³¸Óº¦Â½¤Ñ¤F¤~¹ï.«ç¤Ï¤§¤U¶^
²¦³ºNVÁÙ¬O³Ì±j¤jªºGPU µu´Á¤º¤]µL¤H¥i¥H¶W¶V |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
a2979
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
3222 Posts |
Posted - 02/04/2025 : 10:32:26
|
quote: Originally posted by Light
Deepseek ¬O¾ã¦X²{¤µAI§Þ³N¡A¥[¤p¨B³Ð·s¡A¤£¬O¤°»ò°ò¦¬ð¯}¡F³o¬O²{¤µAIµo®iªº¼Ò¦¡¡A¯¸¦b¥L¤HªÓ»H¤W¿N¿ú¡A¨S¤H¥i§K¡C³Ìªñ³Q§j±o³o»ò¼ö¡A®£¦³¤º¥~«Åªº±À¤â¡C
²Ó¬d¹êÃÒ¡C
1. ¬ð¯}©Ê¥u»Ý¤Q¤À¤§¤@ºâ¤O¡G§j¡C
2023¦~«e¡ALLM¥D¬y¬O dense ¼Ò¦¡¡A«Ü¯Óºâ¤O¡C2023¦~2¤ë¡AGPT-4§ï¥Î MoE ¬Ùºâ¤O¡A2024¦~3¤ë¡AGoogle Gemini 1.5 ¸ò¶i¤]¥Î MoE¡CµM«á2024¦~¥X¤F¤@°ï¶}·½ MoE¡A¥]¬A Deepseek¡C
MoE ¤ñ dense ªºÀu¶Õ¡A´N¦b©ó Na ¤p¡A¥i¬O¨C¤@®a MoE ³£¦³¦P¼ËªºÀu¶Õ¡C©Ò¥H Deepseek ¥u®³ Meta ©M Alibaba ªº dense ¨Ó¤ñ¡A¥i¨S´±»¡¤ñ¨ä¥¦ MoE ¬Ùºâ¤O¡C
2. Deepseek ¶}µo§C¦¨¥»¡G»~¸Ñ¡Ï¥~«Å§j
AI ´N¬O¿N¿ú¡C±ç¤å¾W¦b³Ð¿ì deepseek ®É»¨®ðºÙ¤w¶R¤F¤@¸U¤ùA100¡C
Deepseek V3 »¡ªº 560¸U¬ü¤¸¡A¬O¶]¤@¦¸¥¿¦¡°V½m¯Ó¥ÎH800 GPU ®É¼Æ x $2/¤p®Éªº¯Â¾÷¾¹¶O¥Î¡C¤§«e¬ã¨s¡Bµo®i¼Ò«¬¡B»`¶°°V½m¸ê®Æ¡B¸Õ¶]¡B½Õ¸Õ¡B¥¢±Ñ«¶]µ¥µ¥¶O¥Î³£¤£¦b¤º¡C¤£»¡§Oªº¡A¶]³o55¤Ñ¡AºÞ±±¡BÀˬd¡B½Õ´ú¡BºÊµø¡B×´_ªº¤H¤u³£¤£¥Îªá¿ú°Ú¡H
Meta ªº LLaMA2 70B ¦b2023¦~¶]¤@¦¸¥¿¦¡°V½m¯Ó¥ÎA100 GPU ¦@1.72M¤p®É x $1.5/¤p®É¡A¨º¬O 258¸U¬ü¤¸¡A«ç»ò¨S¤H¸õ¥X¨Ó»¡¦n´Î´Î§C¦¨¥»¡H
DeepSeek-V3 costs only 2.788M GPU hours for its full training. Assuming the rental price of the H800 GPU is $2 per GPU hour, our total training costs amount to only $5.576M. Note that the aforementioned costs include only the official training of DeepSeek-V3, excluding the costs associated with prior research and ablation experiments on architectures, algorithms, or data.
Our main contribution includes:
3. PTX ¶¹L CUDA ³Ð·s¼W¥[ºâ¤O¡G»~¸Ñ + ¥~«Å§j
deepseek ¥Î PTX ¼g³q°Tµ{¦¡¬OµL©`¡A¬°¤F¶¹L H800 NVLink ÀW¼e¶È 400 GB/s ªº¨î¡C¥ĻâGPU¸Ì132ÓSM¸Ì¡A¼·¥X20ÓSM±Mªù¶]PTX¨Ó³B²z³q°T¡C³o¥ß°¨·l¥¢15¢HªºµwÅéºâ¤O¡Cdeepseek¦Û¤v§Þ³N³ø§i¤]µL©`¦a»¡³o¤èªk«D±`¨S¦³®Ä²v¡C
¥i¥H¬Ý¥L̪º TR https://arxiv.org/html/2412.19437v1
In DeepSeek-V3, we implement the overlap between computation and communication to hide the communication latency during computation. This significantly reduces the dependency on communication bandwidth compared to serial computation and communication. However, the current communication implementation relies on expensive SMs (e.g., we allocate 20 out of the 132 SMs available in the H800 GPU for this purpose), which will limit the computational throughput. Moreover, using SMs for communication results in significant inefficiencies, as tensor cores remain entirely -utilized.
4. ¶]¤ÀÀu³Ó¡G¤Ñ¤Ñ§ó·s¡Aťť´N¦n¡C
¬Q¤Ñªü¨½¤Ú¤Ú°¨¤W¸õ¥X¨Ó»¡¥L̪º·s Qwen2.5-Max MoE ¥þ±À£¨î Deepseek¡C©Ò¥H¶]¤À³oºØ¨Æ¡Aťť´N¦n¡A¹L¨â¤Ñ´N·|¦³¤H»¡Ä¹¹Lªü¨½¤Ú¤Ú¡C
5. NVidia §¹³J¡A§C¶¥ GPU ´N¥i½m¦¨ AI¡G°§è
H800 ºâ¤O©M H100 ¤@¼Ë¡A®t§O¦b©ó NVLink BW ³Q¬å¢w¢w 400 GB/s vs 900 GB/s¡C
¤µ¦~§ï¦¨ MoE ªº·|§ó¦h¡A½Ö´±¤£¶R§ó¦n§ó·sªº GPU¡H
¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w¢w ¹ï¾ú¥v¬ã¨sªÌ¦Ó¨¥¡A³Ì¨ë¿EªÌ¡AµL¹L©ó¦b¾ú¥v©M¶Ç»¡¤§¶¡¡A§ä¨ì¤F³Ì§¹¾ãªº¬G¨Æ¡C¢w¢w ºq¼w
¤@ÉE张A100ªä¤ù,仅¬Û当¤_2000张H100¡A§Ú们´N«öºâ¤O¦n¤F¡Aopenai¤â¤W¦³¦h¤Öºâ¤O¡A¬Û当¤_¦h¤Ö张A100¤~·d¥X¦Pý©ªº¦¨ªG¡H
¥B这¤@ÉE张A100¤]«Ddeepseek独¦³¦Ó¬O¥À¤½¥q¤Û¤è¶q¤Æªºªä¤ù¡A¤H®a¥D业¾ã¤Ñ¦£þÓ¥ÎAiªÑ¥«³Î´µæ赚¤j钱¡A¯à挤¥X¦h¤Öºâ¤O给¤l¤½¥q¡H§Y¨Ï撑¦º¤]´N¤@ÉE张A100
ªÑɲ为¤°¤\ÂÛ¡A¦]为单纯¨Ì¾a蛮¤O®Ä²v§C¤U¡A§A¦b这¨½还¦b«÷训练¯Ó¥Î¡A¥X来ªº¦¨ªG¬O¤@ý©ªº吗¡H
¦Ü¤_绕过CUDA这¬O®Ä²v问题吗¡H¦b垄断±«e15%ªººâ¤O¦¨¥»ºâ个¤ò线¡A§ANVidia垄断ªwªj§jªº这¤\¤j¡A蛮¤O¤S¥¢®Äªº±¡úG¤U¡A¤£¶^§A¶^谁 |
Edited by - a2979 on 02/04/2025 10:39:08 |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
damau
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
1329 Posts |
Posted - 02/04/2025 : 14:46:59
|
quote: Originally posted by Henschel
«Xù´µ¦Ñ¤jôª½±¡¦³³ø¾É»¡¤f国deepshit¬O°ò©óĬÁp§Þ³N¶}µo¡A¤f国¸£´Ý«H¤£«H?![](icon_smile_big.gif)
³oÂI¥i¯à©Ê¤£¤p¡A©M¤¤°ê¬Û¤ñ¡AĬÁpªº¼Æ¾Ç¦¨´N·í®É¬O¥@¬É¤W¼Æ¤@¼Æ¤G±jªº¡A¯d¾Ç®É¤@ӱкô¸ôºtºâªº±Ð±Â´N¬O«eĬÁp¥X¨Óªº¼Æ¾Ç®a¡A¥Lªº²z½×Å¥説«Ü¦hºô¸ô¤½¥q¦b¥Î¡A¦³¾Ç¥Í´£¥X½Ð¥L¤¶²Ð²³æ¤¶²Ð¸Ó²z½×¡A¥L¦Ñ¥S¤@®É©Ê°_Á¿¤F¤@¸`½Ò¡Aµ²ªG¤j®a³£Å¥ªºÃúÂÄÂÄ. ¤£ºÞ¬O½Öªº²z½ü¡A¯à¥Î¤W¦³®Ä¤]¬OÓ¤£¤pªº¦¨´N |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
a2979
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
3222 Posts |
Posted - 02/04/2025 : 15:04:35
|
quote: Originally posted by damau
quote: Originally posted by Henschel
«Xù´µ¦Ñ¤jôª½±¡¦³³ø¾É»¡¤f国deepshit¬O°ò©óĬÁp§Þ³N¶}µo¡A¤f国¸£´Ý«H¤£«H?![](icon_smile_big.gif)
³oÂI¥i¯à©Ê¤£¤p¡A©M¤¤°ê¬Û¤ñ¡AĬÁpªº¼Æ¾Ç¦¨´N·í®É¬O¥@¬É¤W¼Æ¤@¼Æ¤G±jªº¡A¯d¾Ç®É¤@ӱкô¸ôºtºâªº±Ð±Â´N¬O«eĬÁp¥X¨Óªº¼Æ¾Ç®a¡A¥Lªº²z½×Å¥説«Ü¦hºô¸ô¤½¥q¦b¥Î¡A¦³¾Ç¥Í´£¥X½Ð¥L¤¶²Ð²³æ¤¶²Ð¸Ó²z½×¡A¥L¦Ñ¥S¤@®É©Ê°_Á¿¤F¤@¸`½Ò¡Aµ²ªG¤j®a³£Å¥ªºÃúÂÄÂÄ. ¤£ºÞ¬O½Öªº²z½ü¡A¯à¥Î¤W¦³®Ä¤]¬OÓ¤£¤pªº¦¨´N
¬¡¤[见¡A«X罗´µ讽¨ë´CÊ^¡mÈDÈUÈcÈdÈfÈUÈbÈU¡n编¥X来ªº¬q¤l¡A¤@个´±传¡A¤@个´±«H![](icon_smile_cool.gif) |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
kingcobra
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
2235 Posts |
Posted - 02/04/2025 : 16:06:14
|
¥H«e¬O¤¤°ê¤H§ä¦UºØ¼Æ¾Ú¡B·s»D ¨ÓÃÒ©ú ¤¤°ê¨S¦³¨º»ò®z¡I ²{¦b¥xµì¾ã¤Ñ§äÃÒ¾Ú¨ÓÃÒ©ú¡A¤¤°ê¨S¦³¨º»ò±j¡I |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
Henschel
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
3386 Posts |
Posted - 02/04/2025 : 16:29:38
|
¨º¤p¬õ®Ñ¡A§Ýµ¤W¤f国¸£´Ý§ä¨Ó¦ÛÂpªº¦UºØ¼Æ¾Ú¡B·s»D¬O?![](icon_smile_wink.gif) |
Edited by - Henschel on 02/04/2025 16:30:04 |
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
guy123
¸ô¤H¥Ò¤A¤þ
![](icon_star_red.gif) ![](icon_star_red.gif) ![](icon_star_red.gif)
2557 Posts |
Posted - 02/04/2025 : 16:32:25
|
«Xù´µ¦Ñ¤jôª½±¡¦³³ø¾É»¡¤f国deepshit¬O°ò©óĬÁp§Þ³N¶}µo¡A¤f国¸£´Ý«H¤£«H?![](icon_smile_big.gif) === ¤f¤f§A¤£¬O国¤å¤£¦æ¡A¦Ó¬O¯uªº¬O脑残¦Ó¤w¡A难©Ç说话«e¨¥¤£·f¦Z语¡C 苏联¦³¤òªºªºAI§Þ术¡A苏联时´Á²¾动¦s储³£¬O1M¤j¤pªº软盘为¥D¡A计ºâ¯à¤O§ó¬OÆÓ¤ñ§C¤U¡C§Ú们现¦b¥Îªº¤âÉó¤ñ80¦~¥Nªº¶W级计ºâÉó运ºâ³t«×³£n5000¿¡C¤£过¤f¤f§A应该¦b¥Î诺°ò亚¡A¤£µM脑¤l¨½为¤°¤\¬O块¥Û头¡C
苏联¦Û¨点错¤F¬ã¨s¤è¦V¡A´¿经顽©Tªº¬ã¨s¯uªÅºÞ¡A结ªG¥b导Ê^时¥N³Q远远¥Ï¨ì¤F¦Z±¡ã¡ã¡ã
¤f国¸£´Ý¬JµMÂà¶iÂåÀø¡A¨º¤£¦p»¡»¡¤f国³Ì·s国®a±ÄÁʪº³Â¾KÃÄ¥i¥HÅý¯f¤H¦b¤â³N¤¤¿ô¨Óªº³ÐÁ| ==== §A这¤f¤f©È¬O¤£ª¾¹D¥þ³Â¡A¥b¨³Â¾K¡A还¦³§½³¡³Â¾K¡H¦Z两ªÌ¤£¨ì¥i¥H¿ô过来¡A还¥i¥H说话¡C
话说¡A¤f¤f§A¯u¤£·\¬O¥¶粪¤u¤H¡A专«õ²ö¦W¨ä§®ªº粪¡C¤f¤f§A¬O§_ª¾¹D¦Ñ¬üªº医疗¨Æ¬G¥i¤ñ¤¤国¦hªº¦h¡A§A¬Ý¬Ý医¥Íªº¦¬¤J¦³¦h¤Ö¬On°ª额ªº执业«O险....医¥Í临§É经验§ó¬O没ªk©M¤¤国¤ñ¡C
|
![Go to Top of Page Go to Top of Page](icon_go_up.gif) |
|
¼ÐÃD ![¤U¤@Ó¼ÐÃD ¤U¤@Ó¼ÐÃD](icon_go_right.gif) |
|