Run a model in gem5-Aladdin simulation¶
Following the tutorial in Build a SMAUG model with Python API, now we are able to create a DL model using the SMAUG Python APIs. In this tutorial, we will proceed to introduce the steps of running a model in gem5-Aladdin simulation. We strongly recommend using our Docker image and this tutorial assumes we are in the Docker image.
Build gem5-Aladdin¶
First, if you haven’t built gem5-Aladdin (cloned in
/workspace/gem5-aladdin
), use the following command to build it.
python2.7 `which scons` build/X86/gem5.opt PROTOCOL=MESI_Two_Level_aladdin -j2
Here, the MESI_Two_Level_aladdin
is the coherence protocol we will use
in the gem5 Ruby memory model. Change -j
parameter to increase the
number of CPU threads to speed up the build, but keep in mind that you may run
out of memory before you run out of CPUs. Running out of memory or disk space
can cause mysterious build failures.
Generate dynamic trace for gem5-Aladdin simulation¶
gem5-Aladdin, which provides the simulation capabilities of estimating performance/power/area for pre-RTL kernels via various SoCs, is a trace-driven simulator. Thus, in SMAUG, in order to run a model in gem5-Aladdin, we need to generate a dynamic trace for the kernels to be simulated as hardware blocks.
First, we need to build the SMAUG tracer, which is an instrumented binary that will be executed to generate the dynamic trace.
make tracer -j4
After running the above command, we will get a binary under build/bin/
named smaug-instrumented
. Then we will run the model created in
Build a SMAUG model with Python API using the following command:
build/bin/smaug-instrumented my_model_topo.pbtxt my_model_params.pb
While the tracer is producing the dynamic trace, we will see outputs such as the following pair for each invocation of the hardware blocks:
dynamic_trace_acc0.gz: Starting to log at inst = 0.
dynamic_trace_acc0.gz: Stopping logging at inst 18199217.
A trace file named dynamic_trace_acc0.gz
is generated.
Note that for large models, this step can take a long time and the resulting trace file can be quite large. Sampling should be used to solve this, which is discussed in Apply sampling to reduce simulation time and trace storage.
Create gem5-Aladdin configuraion files¶
gem5-Aladdin also requires two configuration files for running simulations. One
file specifies SoC-level parameters such as each accelerator’s ID, cache or DMA
configurations, location of the corresponding dynamic trace file, and etc. An
example can found be in experiments/sims/smv/minerva/gem5.cfg
(make
sure the submodules are initialized by running git submodule update
--init --recursive
). The other file gives the parameters to be used for
implementing the accelerator, such as parameters for applying loop unrolling
and scratchpad partition. An example configuration file of implementing the
hardware blocks in our SMV backend can be found in
experiments/sims/smv/smv-accel.cfg
. If you don’t make any changes to
the SMV backend, both of these configuration files are sufficient to simulate
any model run on SMV. You can change the parameters but it may lead to
unexpected accelerator performance characteristics especially if you change
smv-accel.cfg
, as the array partitioning/loop unrolling/loop pipelining
are all fine tuned to achieve a specific throughput.
We defer to the gem5-Aladdin GitHub repo and gem5-Aladdin tutorial for more details of writing these configuration files.
Run the first simulation¶
Assuming we use configuration files in
experiments/sims/smv/tests/minerva/gem5.cfg
and experiments/sims/smv/smv-accel.cfg
, let’s create a folder for our
first simulation. It contains these files:
root@72e3f78202e0:test # ls
dynamic_trace_acc0.gz gem5.cfg my_model_params.pb my_model_topo.pbtxt smv-accel.cfg
Now we are ready to launch the simulation.
/workspace/gem5-aladdin/build/X86/gem5.opt \
--debug-flags=Aladdin,HybridDatapath \
--outdir=outputs \
/workspace/gem5-aladdin/configs/aladdin/aladdin_se.py \
--num-cpus=1 \
--mem-size=4GB \
--mem-type=LPDDR4_3200_2x16 \
--cpu-clock=2.5GHz \
--cpu-type=DerivO3CPU \
--ruby \
--access-backing-store \
--l2_size=2097152 \
--l2_assoc=16 \
--cacheline_size=32 \
--accel_cfg_file=gem5.cfg \
--fast-forward=10000000000 \
-c /workspace/smaug/build/bin/smaug \
-o "my_model_topo.pbtxt my_model_params.pb --gem5 --debug-level=0"
This command runs our custom 3-level DNN model in gem5-Aladdin simulation. gem5-Aladdin provides a wide range of SoC simulation choices, for instance, here, the simulated SoC has an out-of-order CPU running at 2.5GHZ, a two-level cache hierarchy with a 2MB, 16-way associative L2 cache and 32B cacheline size.
There are a few parameters that have special importance:
--mem-size=4GB
will attempt to allocate an enitre 4GB block of memory. If the host machine is low on memory, this could fail.The
--num-cpus=1
can be adjusted to allow for multithreading (see Simulate with multiple threads for discussion).The
--fast-forward
parameter is used to speed up the simulation of the initialization phase, which uses a simplified CPU model in gem5. It is set to a very large cycle value to ensure that gem5 doesn’t automatically switch to detailed CPU too early. In SMAUG, we use gem5’s magic instructionm5_switch_cpu
to switch to the detailed CPU when the initialization is done.For the SMAUG commandline options, note that now we must pass
--gem5
flag to enable gem5-Aladdin simulation.
After the simulation starts, we will see the output look like this:
Model topology file: my_model_topo.pbtxt
Model parameters file: my_model_params.pb
Number of accelerators: 1
info: Increasing stack size by one page.
======================================================
Loading the network model...
======================================================
======================================================
Summary of the network.
======================================================
____________________________________________________________________________________________
Layer (type) Output shape Parameters
____________________________________________________________________________________________
data_2 (Data) (10, 6272) 0
____________________________________________________________________________________________
data_1 (Data) (32, 3, 3, 1) 0
____________________________________________________________________________________________
data (Data) (1, 28, 28, 1) 0
____________________________________________________________________________________________
conv (Convolution3d) (1, 28, 28, 32) 288
____________________________________________________________________________________________
max_pool (MaxPooling) (1, 14, 14, 32) 0
____________________________________________________________________________________________
reorder (Reorder) (1, 6272) 0
____________________________________________________________________________________________
mat_mul (InnerProduct) (1, 10) 62720
____________________________________________________________________________________________
This means the model has been successfully loaded in the simulation. Then we will see the following right after the network summary:
======================================================
Tiling operators of the network...
======================================================
Tiling conv (Convolution3d).
Tiling data (Data).
Tiling data_1 (Data).
Tiling data_2 (Data).
Tiling mat_mul (InnerProduct).
Tiling max_pool (MaxPooling).
Tiling reorder (Reorder).
This shows that SMAUG is performing pre-tiling procedures for each operator - read-only tensors such as weights can be tiled before the actual layer-by-layer network execution. After this, we can see the simulation switches to use the OoO CPU model:
Switched CPUS @ tick 29955086000
switching cpus
This means the initialization is done and SMAUG will start scheduling operators of the model. We can see:
======================================================
Scheduling operators of the network...
======================================================
Scheduling data (Data).
Scheduling data_1 (Data).
Scheduling data_2 (Data).
Scheduling conv (Convolution3d).
As the conv
operator needs to invoke the convolution engine, we begin
to see the Aladdin simulation logs. After each invocation of the hardware
block, results are printed:
===============================
Aladdin Results
===============================
Running : ./outputs/nnet_fwd
Top level function: smv_conv3d_nhwc_vec_fxp
Cycle : 89771 cycles
Upsampled Cycle : 0 cycles
Avg Power: 132.997 mW
Idle FU Cycles: 24432 cycles
Avg FU Power: 111.49 mW
Avg FU Dynamic Power: 102.124 mW
Avg FU leakage Power: 9.36592 mW
Avg MEM Power: 21.5071 mW
Avg MEM Dynamic Power: 2.5961 mW
Avg MEM Leakage Power: 18.911 mW
Total Area: 1.91503e+06 uM^2
FU Area: 775140 uM^2
MEM Area: 1.13989e+06 uM^2
Num of Multipliers (32-bit): 37
Num of Adders (32-bit): 138
Num of Bit-wise Operators (32-bit): 12
Num of Shifters (32-bit): 21
Num of Registers (32-bit): 1798
===============================
Aladdin Results
===============================
Congratulations! You just finished running the first SMAUG simulation. The
outputs
folder contains simulation stats generated by gem5-Aladdin.
How long the simulation will take depends on the machine. On my Intel Core i7 9850H, it takes about 10 minutes. For larger models, both the trace storage and simulation time can become problematic, read on to see the solution.
Apply sampling to reduce simulation time and trace storage¶
To save simulation time and trace storage for large models, we can sample the
loop iterations in the oftentimes repetitive DL kernels. To enable sampling,
use the --sample-level
and --sample-num
parameters in SMAUG.
--sample-level
sets the sampling level, which can be one of the five
levels: no
, low
, medium
, high
and
very_high
. --sample-num
sets the number of iterations a sampled
loop will run. A smaller value means more sampling will be applied. Details
about using the sampling parameters and the gem5-Aladdin sampling APIs are
discussed in Sampling of accelerated kernels.
Simulate with multiple threads¶
If the SoC has multiple CPUs (specified via the --num-cpus
parameter),
we can enable SMAUG’s multithreading feature by passing the
--num-threads
parameter to it. See Multithreading in gem5 SE mode for more details.