diff --git a/docs/mindformers/docs/source_en/feature/start_tasks.md b/docs/mindformers/docs/source_en/feature/start_tasks.md index eb8ddb6c9fbde55a61ac5aae7670a9c9acbd0be4..3ebb7f160021a600d12a5f54454cf9ce970fefa9 100644 --- a/docs/mindformers/docs/source_en/feature/start_tasks.md +++ b/docs/mindformers/docs/source_en/feature/start_tasks.md @@ -145,12 +145,8 @@ Take Qwen2.5-0.5B as an example to perform 2-node 16-device fine-tuning. ```yaml parallel_config: - data_parallel: 2 - model_parallel: 4 - pipeline_stage: 2 - micro_batch_num: 16 - vocab_emb_dp: True - gradient_aggregation_group: 4 + data_parallel: 16 + ... ``` > If the number of nodes and the number of devices are used to change, `data_parallel`, `model_parallel`, and `pipeline_stage` need to be modified to meet the actual number of running devices. `device_num=data_parallel×model_parallel×pipeline_stage`. Meanwhile, `micro_batch_num >= pipeline_stage`. diff --git a/docs/mindformers/docs/source_zh_cn/feature/start_tasks.md b/docs/mindformers/docs/source_zh_cn/feature/start_tasks.md index f1c8e65b40b4aa1a3384b480d32b854e6cc0e14d..82da03c0d8fa947d6652011daf670d24e3052a3f 100644 --- a/docs/mindformers/docs/source_zh_cn/feature/start_tasks.md +++ b/docs/mindformers/docs/source_zh_cn/feature/start_tasks.md @@ -145,12 +145,8 @@ bash scripts/msrun_launcher.sh "run_mindformer.py \ ```yaml parallel_config: - data_parallel: 2 - model_parallel: 4 - pipeline_stage: 2 - micro_batch_num: 16 - vocab_emb_dp: True - gradient_aggregation_group: 4 + data_parallel: 16 + ... ``` > 如使用节点数和卡数改变需要修改`data_parallel`、 `model_parallel`、 `pipeline_stage`满足实际运行的卡数 `device_num=data_parallel×model_parallel×pipeline_stage`,同时满足`micro_batch_num >= pipeline_stage`。