Segmentation Fault in OpenROAD During Multi-Corner STA (sky130A PDK) #6227

weafo · 2024-11-24T15:53:17Z

Describe the bug

OpenROAD crashes with a segmentation fault during the execution of the multi-corner static timing analysis (STA) step in OpenLane. The crash occurs when using the sky130A PDK, and the log reports a segmentation violation in libc. The issue is reproducible across multiple runs with the same configuration.

Steps I do:

Clone the OpenLane repository and set up the environment.
Install the sky130A PDK and set the PDK_ROOT environment variable.
Run the following command:
./flow.tcl -design <your_design> -run_full_flow
The segmentation fault occurs during the STA step in OpenROAD.

Expected Behavior

OpenROAD should complete the multi-corner STA step without crashing and generate timing reports.

OpenROAD Environment

I am using OpenROAD through the OpenLane Docker image: efabless/openlane-tools:openroad_app-cdf98f3571ef892d091b6825c5d473bcfbf6e167-centos-7-amd64 with OpenROAD.

OpenLane Environment

open_pdks cdf98f3571ef892d091b6825c5d473bcfbf6e167
Kernel: 5.15.0-122-generic
Description: Ubuntu 20.04.6 LTS
Docker version 27.3.1, build ce12230
Python 3.11.10
cdf98f35 Thu Jul 18 14:28:46 2024 +0300 Propagate the clock in some STA reports (#2134)
python-venv: INSTALLED

To Reproduce

Clone the OpenLane repository.
Install the sky130A PDK and set PDK_ROOT to point to the installation.
Navigate to the OpenLane directory.
run make mount

Run the following command:

./flow.tcl -design adder_depth_2023 -init_design_config -add_to_designs
./flow.tcl -design adder_depth_2023 -init_design_config -add_to_designs -config_file config.tcl
./flow.tcl -design adder_depth_2023 ```

The segmentation fault occurs during the multi-corner STA step in OpenROAD.

I've also attached a zip of the issue reproducible folder.
issue_reproducible.zip

Relevant log output

OpenLane cdf98f3571ef892d091b6825c5d473bcfbf6e167
All rights reserved. (c) 2020-2023 Efabless Corporation and contributors.
Available under the Apache License, version 2.0. See the LICENSE file for more details.

[INFO]: Using configuration in 'designs/adder_depth_2023/config.tcl'...
[INFO]: Process Design Kit: sky130A
[INFO]: PDK Root: /home/mvguser/.volare
[INFO]: Standard Cell Library: sky130_fd_sc_hd
[INFO]: Optimization Standard Cell Library: sky130_fd_sc_hd
[INFO]: Run Directory: /openlane/designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20
[INFO]: Saving runtime environment...
[INFO]: Preparing LEF files for the nom corner...
[INFO]: Preparing LEF files for the min corner...
[INFO]: Preparing LEF files for the max corner...
[INFO]: Running linter (Verilator) (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/synthesis/linter.log)...
[INFO]: 0 errors found by linter
[INFO]: 0 warnings found by linter
[STEP 1]
[INFO]: Running Synthesis (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/synthesis/1-synthesis.log)...
[STEP 2]
[INFO]: Running Single-Corner Static Timing Analysis (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/synthesis/2-sta.log)...
[STEP 3]
[INFO]: Running Initial Floorplanning (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/floorplan/3-initial_fp.log)...
[INFO]: Floorplanned with width 1488.56 and height 1476.96.
[STEP 4]
[INFO]: Running IO Placement (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/floorplan/4-io.log)...
[STEP 5]
[INFO]: Running Tap/Decap Insertion (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/floorplan/5-tap.log)...
[INFO]: Power planning with power {VPWR} and ground {VGND}...
[STEP 6]
[INFO]: Generating PDN (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/floorplan/6-pdn.log)...
[STEP 7]
[INFO]: Running Global Placement (skip_io) (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/placement/6-global_skip_io.log)...
[STEP 8]
[INFO]: Running Single-Corner Static Timing Analysis (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/placement/8-gpl_sta.log)...
[STEP 9]
[INFO]: Running IO Placement (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/placement/9-io.log)...
[STEP 10]
[INFO]: Running Global Placement (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/placement/9-global.log)...
[STEP 11]
[INFO]: Running Single-Corner Static Timing Analysis (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/placement/11-gpl_sta.log)...
[STEP 12]
[INFO]: Running Placement Resizer Design Optimizations (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/placement/12-resizer.log)...
[STEP 13]
[INFO]: Running Detailed Placement (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/placement/13-detailed.log)...
[STEP 14]
[INFO]: Running Single-Corner Static Timing Analysis (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/placement/14-dpl_sta.log)...
[INFO]: ::env(CLOCK_PORT) is not set
[WARNING]: Skipping CTS...
[STEP 15]
[INFO]: Running Placement Resizer Timing Optimizations (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/cts/15-resizer.log)...
[STEP 16]
[INFO]: Running Global Routing Resizer Design Optimizations (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/routing/16-resizer_design.log)...
[STEP 17]
[INFO]: Running Single-Corner Static Timing Analysis (log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/routing/17-rsz_design_sta.log)...
[ERROR]: during executing openroad script /openlane/scripts/openroad/sta/multi_corner.tcl
[ERROR]: Log: designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/routing/17-rsz_design_sta.log
[ERROR]: Last 10 lines:
12# Tcl_EvalEx in /nix/store/p16s3zg96afj68mfp17m03svhwmdgcz8-tcl-8.6.13/lib/libtcl.so
13# Tcl_Eval in /nix/store/p16s3zg96afj68mfp17m03svhwmdgcz8-tcl-8.6.13/lib/libtcl.so
14# sta::sourceTclFile(char const*, bool, bool, Tcl_Interp*) in openroad
15# ord::tclAppInit(Tcl_Interp*) in openroad
16# Tcl_MainEx in /nix/store/p16s3zg96afj68mfp17m03svhwmdgcz8-tcl-8.6.13/lib/libtcl.so
17# main in openroad
18# 0x00007FEB8734910E in /nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/libc.so.6
19# __libc_start_main in /nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/libc.so.6
20# _start in openroad
child killed: segmentation violation

[ERROR]: Creating issue reproducible...
[INFO]: Saving runtime environment...
OpenLane TCL Issue Packager

EFABLESS CORPORATION AND ALL AUTHORS OF THE OPENLANE PROJECT SHALL NOT BE HELD
LIABLE FOR ANY LEAKS THAT MAY OCCUR TO ANY PROPRIETARY DATA AS A RESULT OF USING
THIS SCRIPT. THIS SCRIPT IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND.

BY USING THIS SCRIPT, YOU ACKNOWLEDGE THAT YOU FULLY UNDERSTAND THIS DISCLAIMER
AND ALL IT ENTAILS.

Parsing config file(s)…
Setting up /openlane/designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/issue_reproducible…
Done.
[INFO]: Reproducible packaged: Please tarball and upload 'designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/issue_reproducible' if you're going to submit an issue.
[ERROR]: Step 17 (routing) failed with error:
-code 1 -level 0 -errorstack {INNER {invokeStk1 throw_error} CALL {run_tcl_script -tool openroad -no_consume /openlane/scripts/openroad/sta/multi_corner.tcl -indexed_log /openlane/designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/routing/17-rsz_design_sta.log -save to=/openlane/designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/results/signoff,noindex,sdf,lib} CALL {run_openroad_script /openlane/scripts/openroad/sta/multi_corner.tcl -indexed_log /openlane/designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/routing/17-rsz_design_sta.log -save to=/openlane/designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/results/signoff,noindex,sdf,lib} CALL {run_sta -propagate_all_clocks -estimate_global -no_save -log /openlane/designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/logs/routing/rsz_design_sta.log} CALL run_resizer_design_routing CALL run_routing CALL run_routing_step CALL {run_non_interactive_mode -design adder_depth_2023}} -errorcode NONE -errorinfo {
    while executing
"throw_error"
    (procedure "run_tcl_script" line 219)
    invoked from within
"run_tcl_script -tool openroad -no_consume {*}$args"
    (procedure "run_openroad_script" line 2)
    invoked from within
"run_$arg_values(-tool)_script $::env(SCRIPTS_DIR)/openroad/sta/multi_corner.tcl {*}$arg_list"
    (procedure "run_sta" line 109)
    invoked from within
"run_sta -propagate_all_clocks -estimate_global -no_save -log $::env(routing_logs)/rsz_design_sta.log"
    (procedure "run_resizer_design_routing" line 13)
    invoked from within
"run_resizer_design_routing"
    (procedure "run_routing" line 8)
    invoked from within
"run_routing"
    (procedure "run_routing_step" line 7)
    invoked from within
"run_routing_step"} -errorline 1
[INFO]: Saving current set of views in 'designs/adder_depth_2023/runs/RUN_2024.11.24_14.53.20/results/final'...
[INFO]: Generating final set of reports...
[ERROR]: Failed to create manufacturability and metric reports:
[ERROR]: Traceback (most recent call last):
  File "/openlane/scripts/generate_reports.py", line 184, in <module>
    cli()
  File "/nix/store/vr8dx2vs7nhz4cz2vfyypx69vbwwrmk5-python3-3.11.9-env/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/vr8dx2vs7nhz4cz2vfyypx69vbwwrmk5-python3-3.11.9-env/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/nix/store/vr8dx2vs7nhz4cz2vfyypx69vbwwrmk5-python3-3.11.9-env/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/vr8dx2vs7nhz4cz2vfyypx69vbwwrmk5-python3-3.11.9-env/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openlane/scripts/generate_reports.py", line 51, in cli
    report = Report(design, tag, design_name, params, run_path).get_report()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openlane/scripts/report/report.py", line 809, in get_report
    *self.extract_all_values(),
     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openlane/scripts/report/report.py", line 359, in extract_all_values
    power_report = Artifact(
                   ^^^^^^^^^
  File "/openlane/scripts/report/report.py", line 62, in __init__
    self.pathname = matches[0]
                    ~~~~~~~^^^
IndexError: list index out of range
[INFO]: Saving runtime environment...
[ERROR]: Flow failed.
[INFO]: The failure may have been because of the following warnings:
[WARNING]: Skipping CTS...

Screenshots

No response

Additional Context

No response

The text was updated successfully, but these errors were encountered:

maliberty · 2024-11-24T20:09:11Z

I can't find that image

Unable to find image 'efabless/openlane-tools:openroad_app-cdf98f3571ef892d091b6825c5d473bcfbf6e167-centos-7-amd64' locally
docker: Error response from daemon: manifest for efabless/openlane-tools:openroad_app-cdf98f3571ef892d091b6825c5d473bcfbf6e167-centos-7-amd64 not found: manifest unknown: manifest unknown.

what version of OpenROAD is in that image? If it is not very current then please try with the head of OR first.

maliberty · 2024-11-24T20:10:09Z

You need to make a self contained test case. The steps to reproduce should be untar & run, not install OL.

weafo · 2024-11-24T23:26:37Z

Sorry for the confusion, I am not using a separate Docker image of OpenROAD. Instead, I use it as a dependency within OpenLane. The version of OpenROAD in my setup is: OpenROAD b16bda7.

maliberty · 2024-11-25T01:46:31Z

That commit is from Jun 3. Please try with the head of OR.

weafo · 2024-12-04T18:38:30Z

I encountered an error while building OpenROAD with the head of OR.
here's the steps:

Edited flake.nix to update the rev

openroad = pkgs.openroad.override {
  rev = "e3ab3fb6e073ec23e499537350008fb2aaaead2e"; 
  sha256 = ""; # Temporarily empty
  openroad-abc = self.openroad-abc;
  opensta = self.opensta;
};

Ran the build command nix build .#openroad --extra-experimental-features nix-command --extra-experimental-features flakes

error: hash mismatch in fixed-output derivation '/nix/store/qyp5zdcziw8arbp02g5ml8wpdba9gw0r-source.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-cYt9Ua9gafWsukB+FBXCuA1leQoXJlDvvfdZtBwGRYI=

Updated flake.nix with the fetched sha256

openroad = pkgs.openroad.override {
  rev = "e3ab3fb6e073ec23e499537350008fb2aaaead2e"; 
  sha256 = "sha256-cYt9Ua9gafWsukB+FBXCuA1leQoXJlDvvfdZtBwGRYI="; 
  openroad-abc = self.openroad-abc;
  opensta = self.opensta;
};

Re-ran the build command nix build .#openroad --extra-experimental-features nix-command --extra-experimental-features flakes

warning: Git tree '/home/mvguser/hdd/OL/OpenLane' is dirty
warning: ignoring untrusted substituter 'https://openlane.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled
error: builder for '/nix/store/k2d3pmaqgg8wmzmbkcqrd238cp4wbabh-openroad.drv' failed with exit code 2;
       last 25 log lines:
       > In file included from /build/source/src/gui/src/gui.cpp:46:
       > In file included from /build/source/src/gui/src/mainWindow.h:43:
       > /build/source/src/gui/src/findDialog.h:46:8: warning: 'accept' overrides a member function but is not marked 'override' [-Winconsistent-missing-override]
       >    46 |   void accept();
       >       |        ^
       > /nix/store/p8cr0k5g2jbjy6k80sdhfyr7vp4zzl8w-qtbase-5.15.12-dev/include/QtWidgets/qdialog.h:97:18: note: overridden virtual function is here
       >    97 |     virtual void accept();
       >       |                  ^
       > In file included from /build/source/src/gui/src/gui.cpp:46:
       > In file included from /build/source/src/gui/src/mainWindow.h:43:
       > /build/source/src/gui/src/findDialog.h:47:8: warning: 'reject' overrides a member function but is not marked 'override' [-Winconsistent-missing-override]
       >    47 |   void reject();
       >       |        ^
       > /nix/store/p8cr0k5g2jbjy6k80sdhfyr7vp4zzl8w-qtbase-5.15.12-dev/include/QtWidgets/qdialog.h:98:18: note: overridden virtual function is here
       >    98 |     virtual void reject();
       >       |                  ^
       > 7 warnings generated.
       > [ 47%] Building CXX object src/gui/CMakeFiles/gui.dir/src/globalConnectDialog.cpp.o
       > 1 error generated.
       > make[2]: *** [src/gui/CMakeFiles/gui.dir/build.make:436: src/gui/CMakeFiles/gui.dir/src/staGuiInterface.cpp.o] Error 1
       > make[2]: *** Waiting for unfinished jobs....
       > 2 warnings generated.
       > 2 warnings generated.
       > make[1]: *** [CMakeFiles/Makefile2:4448: src/gui/CMakeFiles/gui.dir/all] Error 2
       > make: *** [Makefile:136: all] Error 2
       For full logs, run 'nix-store -l /nix/store/k2d3pmaqgg8wmzmbkcqrd238cp4wbabh-openroad.drv'.

Since OpenROAD is used as a dependency, I'm not entirely sure how to properly update it without manually modifying flake.nix. Is there a better approach?

maliberty · 2024-12-04T18:53:03Z

The openroad team does not use flake and the setup is there for openlane usage. @donn any ideas?

donn · 2024-12-04T20:18:38Z

The steps @weafo took were correct, slightly undermined by how the latest version of OpenROAD doesn't build with the flake yet and I have not had the chance to submit a patch for it. Something about boost::optional and fmt.

maliberty added the waiting on op Information has been requested from the Issue Author label Nov 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Segmentation Fault in OpenROAD During Multi-Corner STA (sky130A PDK) #6227

Segmentation Fault in OpenROAD During Multi-Corner STA (sky130A PDK) #6227

weafo commented Nov 24, 2024

maliberty commented Nov 24, 2024

maliberty commented Nov 24, 2024

weafo commented Nov 24, 2024

maliberty commented Nov 25, 2024

weafo commented Dec 4, 2024

maliberty commented Dec 4, 2024

donn commented Dec 4, 2024

Segmentation Fault in OpenROAD During Multi-Corner STA (sky130A PDK) #6227

Segmentation Fault in OpenROAD During Multi-Corner STA (sky130A PDK) #6227

Comments

weafo commented Nov 24, 2024

Describe the bug

Expected Behavior

OpenROAD Environment

OpenLane Environment

To Reproduce

Relevant log output

Screenshots

Additional Context

maliberty commented Nov 24, 2024

maliberty commented Nov 24, 2024

weafo commented Nov 24, 2024

maliberty commented Nov 25, 2024

weafo commented Dec 4, 2024

maliberty commented Dec 4, 2024

donn commented Dec 4, 2024