提交 · v0.5.3 · HPCSource / DeepSpeed

该项目从 https://github.com/microsoft/DeepSpeed 镜像。 Pull mirroring failed 5个月前.
由于尝试失败次数过多，仓库镜像已暂停，可以由项目维护者或所有者恢复。
上次成功更新 5个月前。

9月 16, 2021
- [zero_to_fp32] adapt to 4-bytes alignment in z2 (#1372) · 30537e71
  由 Stas Bekman 创作于 3年前
```
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
```
  v0.5.3
  
  30537e71
- [zero Init] fix regression (#1373) · cf22a691
  由 Stas Bekman 创作于 3年前
```
* [zero Init] fix regression

* clean up the warning
```
  cf22a691
9月 15, 2021
- Introduce a device rank when setting device (#1370) · 90398a70
  由 Sean Naren 创作于 3年前
  
  90398a70
9月 14, 2021
- bump to 0.5.3 · a708b187
  由 Jeff Rasley 创作于 3年前
  
  a708b187
9月 13, 2021
- Update setup.py (#1361) · 8e577c92
  由 Anurag Kumar 创作于 3年前
```
updated classifiers
```
  v0.5.2
  
  8e577c92
9月 10, 2021
- fix unit test failure when merging PP ckpt files (#1359) · 9b915fed
  由 eltonzheng 创作于 3年前
  
  9b915fed
- Remove dropout as client code can do it independently. (#1354) · 9f5939d2
  由 Ammar Ahmad Awan 创作于 3年前
```
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
```
  9f5939d2
- Correctness fix PP+ZeRO for gradient accumulation (#1264) · b712babe
  由 Jeff Rasley 创作于 3年前
```
* pass GAS boundary state from PP -> ZeRO

* formatting

Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
```
  b712babe
- Update website hiring link · bff6126f
  由 Jeff Rasley 创作于 3年前
  
  bff6126f
- update hiring link · 168fce30
  由 Jeff Rasley 创作于 3年前
  
  168fce30
- Update matmul.py (#1349) · eb97a424
  由 Hyunwoong Ko 创作于 3年前
  
  eb97a424
9月 09, 2021

Added 4-byte alignment on NCCL/RCCL (#1328) · 2c628439

由 Aswin John Mathews 创作于 3年前


* Added 4-byte alignment on NCCL/RCCL

* pre-commit formatting fixes

* Fix for checkpoint loading with optimizer partitioning

* Better assert print

* Added unit tests for nccl/rccl 4-byte alignment

* bug

* Updated alignment to implicit

Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

2c628439

[actions] revert unit-test build name · 3e7d06ad
由 Jeff Rasley 创作于 3年前

3e7d06ad

9月 08, 2021
- [actions] add torch version runner label · 0a32c3e2
  由 Jeff Rasley 创作于 3年前
  
  0a32c3e2
9月 02, 2021
- [actions] add master to formatting trigger · 86b948f9
  由 Jeff Rasley 创作于 3年前
  
  86b948f9
- [actions] update branch triggers · 8f299be5
  由 Jeff Rasley 创作于 3年前
  
  8f299be5
- [actions] split formatting and unit tests into two jobs (#1339) · 600db091
  由 Jeff Rasley 创作于 3年前
  
  600db091
9月 01, 2021

Update main.yml (#1338) · 74f058b9
由 Jeff Rasley 创作于 3年前

74f058b9
Use mpu in zero.Init() (#1325) · e08c239c
由 Olatunji Ruwase 创作于 3年前

e08c239c
Support client lr schedulers that are not subclass of torch _LRScheduler (#1337) · 8e301b6e
由 Olatunji Ruwase 创作于 3年前

8e301b6e

Added drop_last to DeepSpeedDataLoader (#1321) · c0b27fb0

由 Hari Prasad 创作于 3年前


* Added drop_last to DeepSpeedDataLoader

This solves issue #326

* Updated drop_last in engine.py

added drop_last as a ds_config as mentioned by @tjruwase

* Update engine.py

* Update engine.py

* updated config.py and constants.py

* Update constants.py

* added dataloader_ prefix

* Update dataloader.py

* corrected yapf test errors

* Update test_data.py

Added dataloader_drop_last unit test

* Corrected yapf and formatting issues

* updated simple_model.py and test_data.py

* Update simple_model.py

* pre-commit fix

* corrected issues

* Update test_data.py

* Update test_data.py

* Update test_data.py

* Update test_data.py

* removed batch_size from test_data.py

* Update simple_model.py

* Update test_data.py

* Update test_data.py

* Fix unit test issues

* Use fp32 to make things work

Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

c0b27fb0

8月 31, 2021
- Remove duplicate clip grad function in deepspeed (#1333) · ddffbae0
  由 Ammar Ahmad Awan 创作于 3年前
```
* Remove the wrong function with duplicate name

* fix format.

* add mpu check. fix tests.
```
  ddffbae0
8月 30, 2021
- update for cuda-11.4 (#1329) · b9ece257
  由 Stas Bekman 创作于 3年前
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  b9ece257
8月 28, 2021
- Use clone to avoid checkpoint bloat (#1326) · 336dd089
  由 Olatunji Ruwase 创作于 3年前
  
  336dd089
8月 27, 2021
- Activation checkpointing improvements (#1254) · 85acf14c
  由 Olatunji Ruwase 创作于 3年前
```
* Rename PA_TO_cpu

* Code cleanup

* Revert accidental change
```
  85acf14c
- use scalar cpu-adam in case of exception in builder (#1259) · 9645e7bc
  由 Reza Yazdani 创作于 3年前
```
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
```
  9645e7bc
- Add more synchronizations and barriers for the multi-gpu inference case (#1309) · 0ec11daa
  由 Reza Yazdani 创作于 3年前
```
* add more synchronizations and barriers for resolving gpu-halt issue

* removing unuseful broadcasts
```
  0ec11daa
8月 26, 2021
- bump to 0.5.2 · c1b0a4e1
  由 Jeff Rasley 创作于 3年前
  
  c1b0a4e1
- Reducing the memory-overhead of creating model for multi-GPU run (#1244) · 49b6a632
  由 Reza Yazdani 创作于 3年前
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  v0.5.1
  
  49b6a632
8月 25, 2021

Support Callable type for client optimizer and lr_scheduler (#1316) · 274c375c

由 Olatunji Ruwase 创作于 3年前

* Callable option for optimizer and scheduler

* Add unit test

* Formatting

* Disable debug prints

* Use base optimizer to construct lr scheduler

* Formatting

* Remove dead import

274c375c

[zero] restore fp16 params if no zero ckpts available (#1322) · aa121291
由 Jeff Rasley 创作于 3年前
```
* restore fp16 params if no zero ckpts available

* formatting
```
aa121291

8月 19, 2021
- Add issue templates · 10b48405
  由 Jeff Rasley 创作于 3年前
  
  10b48405
8月 18, 2021
- [docs] update moe features and news post · 6cd5f87b
  由 Jeff Rasley 创作于 3年前
  
  6cd5f87b
- add MoE press release links · 058ab819
  由 Jeff Rasley 创作于 3年前
  
  058ab819
- Updating the torch version check to numeric (#1314) · e804f150
  由 Pruthvi Madugundu 创作于 3年前
  
  e804f150
8月 17, 2021

add moe to sidebar · e070a09a
由 Jeff Rasley 创作于 3年前

e070a09a
MoE read the docs update (#1312) · 9cb64a1f
由 Jeff Rasley 创作于 3年前

9cb64a1f
bump 0.5.1, DSE, moe docs · a1de767a
由 Jeff Rasley 创作于 3年前

a1de767a

DeepSpeed MoE (#1310) · f2843244

由 Ammar Ahmad Awan 创作于 3年前


Co-authored-by: Alex Muzio <Alex.Muzio@microsoft.com>
Co-authored-by: Ammar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: Conglong Li <conglong.li@gmail.com>
Co-authored-by: Felipe Cruz Salinas <Andres.Cruz@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <shaden.smith@microsoft.com>
Co-authored-by: Young Jin Kim <youki@microsoft.com>
Co-authored-by: bapatra <bapatra@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <shaden.smith@microsoft.com>
Co-authored-by: Young Jin Kim <youki@microsoft.com>

v0.5.0

f2843244

8月 16, 2021

Curriculum learning (#1307) · b2b34ae3

由 Conglong Li 创作于 3年前


Co-authored-by: Conglong Li <conglong.li@gmail.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

b2b34ae3