PyAMG Error during SU2-PyAMG run

[aydinmirac]

Hello everyone,

We are trying to use SU2-pyamg on the server.
We followed the installation steps on SU2 website and did not face any compilation error.

Steps we followed:

1. Download the file with git ("Install a custom version of SU2" section in https://pyamg.saclay.inria.fr/pyamg.html)
2. cd SU2
3. ./bootstrap
4. ./configure --prefix=$HOME/SU2 --enable-mpi --with-cc=/opt/intel/compilers_and_libraries_2019.5.281/linux/mpi/intel64/bin/mpicc --with-cxx=/opt/intel/compilers_and_libraries_2019.5.281/linux/mpi/intel64/bin/mpicxx CXXFLAGS="-O3"
5. make -j 8
6. make install

Then we added the following lines to bashrc file:

export PYTHONPATH="/usr/lib64/python2.7/site-packages"
export SU2_RUN="$HOME/su2-installed/bin"
export SU2_HOME="$HOME/SU2"
export PATH=$PATH:$SU2_RUN
export PYTHONPATH=$PYTHONPATH:$SU2_RUN

We downloaded example file from pyamg website:
https://pyamg.saclay.inria.fr/pyamgexamples.html (Adaptive 3D ONERA M6 wing example)

During running, we encountered the following error:

$HOME/su2-installed/bin/mesh_adaptation_amg.py -f adap_ONERAM6.cfg -n 4 > out.txt &
[1] 166739
[user@host pyamg-su-test]$ Traceback (most recent call last):
File "/home/eduline/su2-installed/bin/mesh_adaptation_amg.py", line 111, in
main()
File "/home/eduline/su2-installed/bin/mesh_adaptation_amg.py", line 72, in main
options.save_all )
File "/home/eduline/su2-installed/bin/mesh_adaptation_amg.py", line 101, in mesh_adaptation_amg
SU2.run.amg(config)
File "/home/eduline/su2-installed/bin/SU2/run/amg.py", line 212, in amg
pyamg.su2_to_libmeshb(config_amg['adap_back'], "", "amg_back")
UnboundLocalError: local variable 'pyamg' referenced before assignment

Python version is 2.7.5 and we installed pyamg by running "sudo pip install pyamg".

Any help would be appreciated.

Best regards,

Mirac

[bmunguia]

Hi Mirac,

The pyamg website is based on an fork of Vic's that's very out of date. The branch feature_adap handles all the pyamg and GMF I/O library installation if you just follow the usual installation instructions (either using preconfigure.py or the meson scripts) in the v7 instructions. Just keep in mind that it's a bit experimental still. The most up-to-date version of pyamg is included, but note that it requires python 3.7 or greater.

Brian

[antares190]

Hi @bmunguia ,
I was in the same situation as @miracaydin1, so thanks for the hint. Nevertheless I'm having some problems. I followed your instructions to install the branch:

1. downloaded the feature_adapt_sst branch.
2. Used meson to configure the build
   ./meson.py build -Denable-autodiff=true -Denable-directdiff=true
3. Added the environment variables to the .bashrc
4. ninja build.
   ./ninja -C build install

The build didn't show any errors, only few warnings during meson ( gcc1: warning: command line option ‘-Wno-non-virtual-dtor’ is valid for C++/ObjC++ but not for C).

When I try to run the mesh_adaption (or even run simply the solver through parallel_computation.py):

$SU2_RUN/mesh_adaptation_amg.py -f invCyl.cfg -n 6

It immediately shows an error (without even showing the SU2 splash screen) :
Traceback (most recent call last):
File "/usr/local/bin/mesh_adaptation_amg.py", line 38, in <module>
import SU2
File "/usr/local/bin/SU2/__init__.py", line 14, in <module>
from SU2 import amginria
File "/usr/local/bin/SU2/amginria/__init__.py", line 4, in <module>
from .interface import *
File "/usr/local/bin/SU2/amginria/interface.py", line 41, in <module>
import _amgio as amgio
ImportError: No module named _amgio

Am I missing any dependencies?
Please note that I previously compiled SU2 master branch without issues, and have already installed mpich, numpy, scipy.
Any help would be really appreciated.
Thanks

[bmunguia]

The _amgio extension should be built by default. I haven't had any issues on any of the machines I've built on, so I appreciate the feedback.

Just curious, are you running with python >=3.7? The shebang in all the SU2 python scripts

#!/usr/bin/env python

uses whatever python is set to in your environment variables, but pyamg/_amgio will only build/run with python >=3.7. Could you also check if the _amgio extension was installed in your site-packages (probably located in ~/.local/lib/python3.x/site-packages)?

If it didn't build/install, you could go into extensions/AMGIO/su2io and run

python3 setup.py build_ext && python3 setup.py install

If it did build/install, try the command

python3 $SU2_RUN/mesh_adaptation_amg.py -f invCyl.cfg -n 6

[antares190]

Hi @bmunguia ,
thanks for the fast reply, I'm sorry, the Python version was 3.6. I installed the 3.7.5, and made sure to reconfigure (had an issue with the gnome-terminal, but solved).
I re-compiled everything, but unfortunately the error persists, so I checked .local/python3.7.5./site-packages/pyamg but no _amgio extension was there.
When I move in external/AMGIO/su2io/ and run :
python3 setup.py build_ext && python3 setup.py install
An error occours:

running build_ext
building '_amgio' extension
swigging ./sources/Python3.7/amgio_py.i to ./sources/Python3.7/amgio_py_wrap.c
swig -python -o ./sources/Python3.7/amgio_py_wrap.c ./sources/Python3.7/amgio_py.i
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.7m -c ./sources/Python3.7/amgio_py.c -o build/temp.linux-x86_64-3.7/./sources/Python3.7/amgio_py.o -std=c99 -Wno-unused-variable -Wno-unused-result -O3
./sources/Python3.7/amgio_py.c:2:10: fatal error: Python.h: No such file or directory
 #include "Python.h"
          ^~~~~~~~~~
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Shall I remove python2.7 as well?
Thanks

During compilation the following warning appears:

cc1: warning: command line option ‘-Wno-non-virtual-dtor’ is valid for C++/ObjC++ but not for C
../externals/AMGIO/sources/libmeshb7.c: In function ‘GmfGetLinTab’:
../externals/AMGIO/sources/libmeshb7.c:1391:35: warning: format ‘%lld’ expects argument of type ‘long long int *’, but argument 3 has type ‘int64_t * {aka long int *}’ [-Wformat=]
             safe_fscanf(msh->hdl, "%lld", &IntTab[ (*IntCpt)++ ], msh->err);
                                   ^       ~~~~~~~~~~~~~~~~~~~~~~
../externals/AMGIO/sources/libmeshb7.c:475:23: note: in definition of macro ‘safe_fscanf’
       if( fscanf(hdl, format, ptr) != 1 ) \
                       ^~~~~~
../externals/AMGIO/sources/libmeshb7.c: In function ‘GmfSetLinTab’:
../externals/AMGIO/sources/libmeshb7.c:1462:35: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 3 has type ‘int64_t {aka long int}’ [-Wformat=]
             fprintf(msh->hdl, "%lld ", LngTab[ LngCpt++ ]);
                                ~~~^    ~~~~~~~~~~~~~~~~~~
                                
%ld

Moreover later on the script (even if it asks me for permissions, and I provide the password):

Installing _amgio.
Installation failed due to insufficient permissions.
Attempting to use polkit to gain elevated privileges...
Installing _amgio.
Installing pyAMG for Linux.
pyAMG installation failed

[bmunguia]

I just updated the build to use the --user flag so it shouldn't require permissions. Can you pull and try installing again? Just run the same meson command as before with --reconfigure

As for the missing header, I think that's an issue with your python setup. Don't remove python2.7 from your system, since a lot of packages still depend on it. How did you install python? If you run locate Python.h does it return anything?

[antares190]

Thanks for the commit, I've installed it using your hints (I also changed some aliases to always point to python3), and now parallel_computation.py it is running properly.
mesh_adaption_amg.py runs, until it complains about Ncorners in the SU2 mesh.
## WARNING: MISSING SU2 MESH FILE KEYWORD: NCORNERS=.
and then mesh_adapt fails, I can't find any reference to NCORNERS in *.su2 mesh files by the way.

As far as locate Python.h that's the output :

/home/antares/Downloads/SALOME-9.3.0-UB18.04-SRC/BINARIES-UB18.04/Python/include/python3.6/Python.h
/home/antares/Downloads/SALOME-9.3.0-UB18.04-SRC/BINARIES-UB18.04/gmsh/include/FieldPython.h
/home/antares/Downloads/SALOME-9.3.0-UB18.04-SRC/BINARIES-UB18.04/gmsh/include/simpleFunctionPython.h
/home/antares/Downloads/ThirdParty-v1806/ParaView-v5.5.2/VTK/Utilities/Python/vtkPython.h
/home/antares/OpenFOAM/ThirdParty-v1806/ParaView-v5.5.2/VTK/Utilities/Python/vtkPython.h
/usr/include/python3.6m/Python.h

Thanks

[bmunguia]

The corners warning is fine. I added the ability to specify corner points for sharp features that the user wants to maintain. For example, in an airfoil test case I want to make sure the sharp trailing edge at node 512 is maintained, so I can manually add

NCORNERS= 1
1 512

to the end of my mesh. It's not necessary, though, and AMG performs ridge detection to identify any features anyway.

I assume since it built and (kinda) ran that you managed to fix your python setup through your aliases?

What was the specific error that you got? I probably should have also mentioned that I've changed a lot of options since updating from Vic's old code. Here are the list of required options (values from my test case):

% Type of sensor used for adaptation
% Options include GOAL (adjoint-based), MACH, PRES
PYADAP_SENSOR= GOAL
% Objective function used for goal-oriented adaptation
OBJECTIVE_FUNCTION= LIFT
% Maximum cell size for adaptation
PYADAP_HMAX= 500.0
% Minimum cell size for adaptation
PYADAP_HMIN= 1e-9
% Gradation factor (typically 1.2-1.8)
PYADAP_HGRAD= 1.8
% Norm used for adaptation
% 1.0 or 2.0 recommended for inviscid flows
% 4.0 recommeneded for viscid
ADAP_NORM= 4.0
% Approximate mesh size (NPOI) at each level
PYADAP_COMPLEXITY= (30000, 60000, 120000)
% Number of adaptations performed at each level
PYADAP_SUBITE= (3, 3, 3)

Some optional options:

% Numerical method for Hessian reconstruction in goal-oriented adaptation
% Currently only Green-Gauss
NUM_METHOD_HESS= GREEN_GAUSS
% Value for ITER for the flow solver at each level
PYADAP_FLOW_ITER= (2999, 4999, 6999)
% Value for ITER for the adjoint solver at each level
PYADAP_ADJ_ITER= (2999, 4999, 6999)
% Value for CFL_NUMBER for the adjoint solver at each level
PYADAP_CFL= (10.0, 15.0, 20.0)
% Background mesh containing fine surface representation
% Can be either a .su2 or .meshb (GMF) file
% If not specified, the baseline mesh is used
PYADAP_BACK= rae2822_euler_40960_split.meshb
% Invert the background mesh orientation
% Sometimes required for 3D meshes, since AMG doesn't perform
% a check that the orientation of the background and foreground
% meshes match
PYADAP_INV_BACK= NO
% Don't use ridge detection if corners specified
PYADAP_RDG= NO

[bmunguia]

FYI, just so I can keep debugging the RANS adaptation in feature_adap_sst, I've merged it into feature_adap. You should use feature_adap instead from here on out since feature_adap_sst will be pretty volatile. I'll modify my comment above in case anybody references this thread.

[antares190]

Thanks for the reply, I'll use feature_adap. Also very appreciated the required and optional command list.
As far as the run is concerned, it exits during the first mesh adaption (no error on the console though). The amg.out is quite long and there are a lot of info.

Let me briefly summarize the case I'm testing:

• Supersonic inviscid 2D axialsymmetric cylinder.
• mesh adaption Mach based.
• no shape optimization.
• mesh adaption (PYADAP_FLOW_ITER= (2999, 4999, 6999))
• coarse mesh (PYADAP_COMPLEXITY= (20000, 60000, 120000))
• ITER = 10000.

I would expect then the solver to start, reach iteration number 2999, adapt the flow mesh, then run until 4999, adapt, 6999 adapt, reach 10000 iterations.
In the :
adapt/ini/log.out
as far as I can tell it reaches 10000 iterations, which is quite unexpected, maybe my problem arise due to mismatching options in the *.cfg file, is there any applicable *.cfg file to start from?

[bmunguia]

Yeah the config is a bit confusing. The initial CFD run (in adap/ini) uses the value specified in ITER (unless RESTART_SOL= YES, then it runs one iteration just to get force coefficients).

After that, it runs a series of adaptations based on the settings in PYADAP_SUBITE, PYADAP_FLOW_ITER, and PYADAP_COMPLEXITY.

1. AMG interpolates the solution
2. A simulation is run for PYADAP_FLOW_ITER[0] iterations (2999 in your case)
3. An adaptation is performed with a target size of PYADAP_COMPLEXITY[0] (20000 in your case)

This is done PYADAP_SUBITE[0] times. Then it moves on to the next mesh level.

I admit that I haven't really debugged the feature-based (e.g. Mach-based) adaptation much. I found a couple bugs in the python scripts and just updated them, so could you pull again and rerun ninja?

The AMG output is pretty long, but since it's developed at Inria, I don't have any control over that. I guess as long as none of the lines are error statements (usually complaining about invalid projections or negative eigenvalues of the metric from my experience) it should be fine. The only actual information I get from amg.log is usually in the last line

NbSensor 30518 msh->npobk 30518 NbrTyp 19 msh->npoin 52477

which just tells you how many points are in the old vs. new mesh, but you could just look at the .su2 files for that.

Another important note is that the python version of AMG currently only supports triangle and tetrahedral volume elements.

[antares190]

Sorry for the delay, I managed to clone the feature_adap branch, and updated it.

Now during the computation I think it manages to go through the first adap, and creates the ite0 folder so basically my understanding is that right now it is working (still have some convergence problems).

Is this branch providing similar convergence rates as SU2 7.0.3 ?

Thank you very much for the support!

[bmunguia]

It's basically 7.0.3, but I've modified the gradient correction, viscous flux Jacobian, and pretty much the entire SST solver. Are you using any of those? If you give me more info about your case and the convergence problems, I can look into it

[antares190]

Thanks Brian,
my bad, I misinterpreted the carpets / logs structure. I successfully ran a test of a very simple case. In the results there are some small artifacts, but the shocks are resolution improved consistently after mesh adaption.
Thanks for the support

[juanjosealonso]

Hi @antares190, Glad to see that things seem to be starting to work for you. We (Brian) are trying to improve this capability in the solver and any experiences / suggestions / help will be welcome. With that said, this seems like an interesting result. Would you mind submitting it (or other pictures that you like better) so the SU2 Foundation can use it to show this capability to others in the future? The link where you can do this is here<http://su2foundation.org/su2-promotional-material/?utm_source=hs_email&utm_medium=email&utm_content=76584389&_hsenc=p2ANqtz-9fEq2awKk2vd155cCcN_N4mWBCZK-rJ-TqNsZhqSJs-VWn-w7q-H6w8sdiA3LyuOTqlK4eqZhFFstKi-LQFyyGqYPdkwYc9JsNdB1yyd7pqMwTFzA&_hsmi=76584389>. Thanks a lot, Juan On Apr 29, 2020, at 1:38 PM, antares190 <notifications@github.com<mailto:notifications@github.com>> wrote: Thanks Brian, my bad, I misinterpreted the carpets / logs structure. I successfully ran a test of a very simple case. In the results there are some small artifacts, but the shocks are resolution improved consistently after mesh adaption. Thanks for the support [Comparison]<https://user-images.githubusercontent.com/41752169/80644172-84a7a580-8a69-11ea-9f16-58e98c808b73.png> [ComparisonMesh]<https://user-images.githubusercontent.com/41752169/80644174-85403c00-8a69-11ea-819f-2ac00550a089.png> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#948 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AA5FFRCGLBHJCB3FS4JSSV3RPCF5RANCNFSM4MQACOXA>.