technical:recipes:gnnunlock

This is an old revision of the document!

Installing GNNUnlock on Caviness

The GNNUnlock Python code (available on GitHub) makes use of the GraphSaint Python code (available on GitHub) which makes use of TensorFlow and its underlying dependencies (like numpy, scipy). GNNUnlock is pure Python code (with two Perl helpers) but GraphSAINT includes compiled components. The recipe is thus:

  1. Create a Python virtual environment with the required TensorFlow dependencies
  2. Compile the GraphSAINT binary components in the virtual environment
  3. Optionally compile the GraphSAINT C++ training program
  4. Create a VALET package definition to manage the GNNUnlock virtual environment(s)

In the resulting virtual environment the following tasks can be performed:

  • Download e.g. Reddit training data from Google drive and process with the C++ training program
  • Convert the example GNNUnlock circuits data to graph format for GNNUnlock
  • Perform GNNUnlock training with converted circuits data

Create the virtual environment

The GNNUnlock C++ program is best-compiled using Intel compilers and the MKL library. Intel oneAPI includes conda for virtual environment tasks so it will be used for the creation of the virtual environment, etc.

In this recipe a versioned software directory hierarchy will be created for GNNUnlock in the user's home directory. This procedure can be modified to install to an alternative location by altering the value assigned to the GNNUNLOCK_PREFIX environment variable. Since GNNUnlock has no releases, branches, or tags present in GitHub, the date of download is adopted as the version. 

[frey@login01.caviness ~]$ GNNUNLOCK_PREFIX=~/sw/gnnunlock
[frey@login01.caviness ~]$ GNNUNLOCK_VERSION=2024.07.01
[frey@login01.caviness ~]$ vpkg_require intel-oneapi/2024
[frey@login01.caviness ~]$ rm -rf ~/.conda/cache
[frey@login01.caviness ~]$ conda create --prefix "${GNNUNLOCK_PREFIX}/${GNNUNLOCK_VERSION}" \
    --override-channels --channel intel --channel anaconda \
    python'>=3.6.8' \
    tensorflow'=1.15.2' \
    cython'>=0.29.2' \
    numpy'>=1.14.3' \
    scipy'>=1.1.0' \
    scikit-learn'>=0.19.1' \
    pyyaml'>=3.12'
[frey@login01.caviness ~]$ conda activate "${GNNUNLOCK_PREFIX}/${GNNUNLOCK_VERSION}"

The conda cache is removed to prevent existing downloaded packages from interfering with what's available online and to keep the user's home directory from growing too large. Restricting the virtualenv creation to only the intel and anaconda channels keeps the package-solving easier and biases toward consuming Intel-based packages that are likely very well-optimized for Caviness' hardware.

Clone the source code

The source repositories for both GNNUnlock and GraphSAINT will be cloned into the virtualenv directory itself, starting with GNNUnlock: 

[frey@login01.caviness ~]$ cd "${GNNUNLOCK_PREFIX}/${GNNUNLOCK_VERSION}"
[frey@login01.caviness 2024.07.01]$ git clone https://github.com/DfX-NYUAD/GNNUnlock.git
[frey@login01.caviness 2024.07.01]$ cd GNNUnlock

The examples presented in the GNNUnlock documentation assume that GraphSAINT has been cloned as a sub-directory of the GNNUnlock directory: 

[frey@login01.caviness GNNUnlock]$ git clone https://github.com/GraphSAINT/GraphSAINT.git
[frey@login01.caviness GNNUnlock]$ pushd GraphSAINT
[frey@login01.caviness GraphSAINT]$

At this point the GraphSAINT repository is the current working directory.

Build GraphSAINT binary components

GraphSAINT includes several binary (cython) components that must be compiled in the current virtualenv. The GraphSAINT (and GNNUnlock) documentation provide the necessary command: 

[frey@login01.caviness GraphSAINT]$ python graphsaint/setup.py build_ext --inplace

The binary components are installed in the graphsaint directory itself, where the Python code expects to find them.

Build the C++ training program

The ipdps19_cpp sub-directory contains the source code for the C++ training program. There are two make files:

  • makefile uses Intel (pre-oneAPI) C++ and the MKL library for linear algebra
  • makefile.nomkl uses GNU C++ and embedded linear algebra functionality that may or may not be parallelized with OpenMP

Since we wish to use Intel oneAPI compilers and MKL, the makefile will be used in slightly altered form. A patch file is supplied for this purpose – download makefile.oneapi.patch and copy it to the ${GNNUNLOCK_PREFIX}/${GNNUNLOCK_VERSION}/GNNUnlock/GraphSAINT directory created in this recipe:

makefile.oneapi.patch
--- A/makefile	2024-07-01 10:09:30.696062752 -0400
+++ B/makefile	2024-07-01 10:12:32.685255418 -0400
@@ -1,8 +1,8 @@
-CC=icc
+CC=icpx
 IDIR=./include
 ODIR=./obj
-LIBS=-L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 
-CFLAGS=-I${IDIR} -I${MKLROOT}/include -fopenmp -pthread -Wall -O3 --std=c++11
+LIBS=
+CFLAGS=-I${IDIR} -qmkl=parallel -qopenmp -pthread -Wall -O3 --std=c++11
 
 _DEPS=global.h optm.h		# global dependencies
 DEPS=$(patsubst %,$(IDIR)/%,$(_DEPS))

The patch gets applied in the source directory: 

[frey@login01.caviness GraphSAINT]$ pushd ipdps19_cpp
[frey@login01.caviness ipdps19_cpp]$ patch -p1 < ../makefile.oneapi.patch
[frey@login01.caviness ipdps19_cpp]$ make
[frey@login01.caviness ipdps19_cpp]$ install train "${GNNUNLOCK_PREFIX}/${GNNUNLOCK_VERSION}/bin/ipdps19-train"
[frey@login01.caviness ipdps19_cpp]$ popd
[frey@login01.caviness GraphSAINT]$ popd
[frey@login01.caviness GNNUnlock]$

The compiled program is installed in the bin directory for the virtualenv as ipdps19-train; when the virtualenv is activated, the program can be executed with the bare command ipdps19-train.

At this point the working directory has been returned to the GNNUnlock repository.

VALET package definition

Before going any further, a VALET package definition file should be created to facilitate the use of GNNUnlock in the future. Since this recipe has created the virtualenv in the user's home directory, it makes sense to create the VALET package definition file therein, as well. For other installation locations (like workgroup storage) an alternative location may be appropriate for the package definition file.

I first note the value of several environment variables used in this recipe: 

[frey@login01.caviness GNNUnlock]$ echo $GNNUNLOCK_PREFIX
/home/1001/sw/gnnunlock
 
[frey@login01.caviness GNNUnlock]$ echo $GNNUNLOCK_VERSION
2024.07.01

Recall that intel-oneapi/2024 was added to the environment at the beginning of this recipe: that is the sole dependency associated with this GNNUnlock version. The VALET package definition file created at ~/.valet/gnnunlock.vpkg_yaml would look like this, with the appropriate value of $GNNUNLOCK_PREFIX substituted for «GNNUNLOCK_PREFIX» etc.:

gnnunlock.vpkg_yaml
gnnunlock:
    prefix: «GNNUNLOCK_PREFIX»
    description: "Graph Neural Networks-based Oracle-less Unlocking Scheme for Provably Secure Logic Locking"
    url: "https://github.com/DfX-NYUAD/GNNUnlock"
    
    versions:
        "«GNNUNLOCK_VERSION»":
            description: sources cloned from github 2024 July 01
            dependencies:
                - intel-oneapi/2024
            actions:
                - action: source
                  script:
                      sh: intel-python.sh
                - variable: GNNUNLOCK_DIR
                  value: ${VALET_PATH_PREFIX}/GNNUnlock
                - variable: GRAPHSAINT_DIR
                  value: ${VALET_PATH_PREFIX}/GNNUnlock/GraphSAINT
  • technical/recipes/gnnunlock.1719855864.txt.gz
  • Last modified: 2024-07-01 13:44
  • by frey