Skip to content

WiP: New OpenACC features for GPUFORT runtime #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
domcharrier wants to merge 76 commits into
base: main
Choose a base branch
from
Open

WiP: New OpenACC features for GPUFORT runtime #29

domcharrier wants to merge 76 commits into from

Conversation

@domcharrier
Copy link
Collaborator

@domcharrier domcharrier commented Oct 28, 2021 •
edited
Loading

FEATURES:

  • Initial support for acc declare

  • OpenACC (gpufort runtime)

    • offloaded loops:
      • add default clause handling
      • default strategy is present_or_copy if neither default(none) nor default(present) s specified.
    • Initial support for acc declare (module/subroutine/function/program variables, fixed-size, allocatable, pointer)

  • Add interoperable GPUFORT array datatype (up to 7 dimensions; autogenerated):

    • Manage host and device pointer pair (can be null)
      • Either wrap exisiting pointers or
      • Allocate (pinned) host memory requested
      • Allocate device memory if requested
    • Provide H2D, D2H copy operations
    • Encode bounds and sizes of an Fortran array
    • Can be configured to perform H2D, D2H copy operations at init/destruction
    • In C++, equipped with operator()(int i1, int i2, ...) to support Fortran style array indexing
      in C++ code. No index macros.

  • Will be used by GPUFORT to construct interoperable derived types from non-interoperable device types.
    This will allow AoS syntax such as:
    domains(5).cells(i).coord_x in GPUFORT C++ code, which is analogous to the Fortran equivalent:
    domain(5)%cells(i)%coord_x.

BUGFIXES:

  • Fix crash when encountering top-level subroutines/functions.

@domcharrier
Start stability test suite for full app.
9a6e6ef
@domcharrier
fort2hip/scanner: Move translator calls into scanner
61efb5c 
* Reason: Need to know all variables in loop kernel
body to generate `present_or_\(copy\|copyin\) runtime calls
for the vars not appearing in clauses.
@domcharrier
Initial implementation of default clause behaviour
12e4fae 
*If no default clause is specified, present_or_copy is performed for all
 unmapped variables
*If 'default(none)' is specified and not all variables are mapped, warning
 is posted (will add option to convert to error)
*If 'default(present)' is specified,

TODO: Take parent data directive into account to prevent
some unnecessary runtime calls (if current behaviour is performance issue).
@domcharrier
WiP on acc declare for programs/subprograms (unstable)
fc316c1
@domcharrier
WiP: More drafting on declare (unstable)
5845a4f
@domcharrier
WiP towards declare (unstable).
8360355
@domcharrier
Merge branch 'main' into develop-acc
9492495
@domcharrier
Merge branch 'main' into develop-acc
9277f10
@domcharrier
Merge branch 'main' into develop-acc
9c79f42
@domcharrier
WiP towards acc declare in subroutines (unstable)
3e15474 
* vector-add-declare/vector-add.f90 example with declare in program seems
  to work correctly
* Some more care required for enabling declare in subroutines
@domcharrier
WiP on declare: Add working example with declare in subroutine
8de6ead
@domcharrier
WiP on acc declare wrt allocatable/pointer vars
fb25e6b
@domcharrier
WiP on module var declare (unstable)
e9c4ea1
@domcharrier
WiP on acc declare (unstable)
3d54735
@domcharrier
WiP on acc declare (unstable,testing)
ff23395
@domcharrier
WiP on acc declare: Add example (unstable)
a0429d3
@domcharrier
WiP on declare output formatting
b15d170
@domcharrier
WiP on declare output formatting
5775e5e
@domcharrier
Merge branch 'develop-acc' of github.com:RocmSoftwarePlatform/gpufort…
4440458 
… into develop-acc
@domcharrier
WiP/gpufort acc runtime: reuse buffers & make tuneable
d5803da 
FEATURE: gpufort acc runtime behavior can now be influenced/
 tuned via the following environment variables:
 GPUFORT_LOG_LEVEL                    (default=0)
   log level. Maximum log level used in code 3.
 GPUFORT_MAX_QUEUES                   (default=64)
   maximum number of async queues.
 GPUFORT_INITIAL_RECORD_LIST_CAPACITY (default=4096)
   mapping records are managed via a vector. Specify
   the initial vector capacity via this flag.
   If the maximum capacity is reached, the vector
   capacity is doubled.
 GPUFORT_BLOCK_SIZE                   (default=32)
   all device arrays are allocated as multiples of
   this block size.
 GPUFORT_REUSE_THRESHOLD              (default=0.9)
   reuse an existing device buffer if the requested
   buffer size greater than GPUFORT_REUSE_THRESHOLD x size of
   already existing buffer.
 GPUFORT_NUM_REFS_TO_DEALLOCATE       (default=-5)
   number of references for which a released array
   will be deallocated

OPTIMIZATION: gpufort acc runtime will now try to
reuse device arrays that have been previously
released but not allocated yet. Behavior can be tuned
via env. vars. GPUFORT_REUSE_THRESHOLD,
GPUFORT_NUM_REFS_TO_DEALLOCATE and in some sense via
GPUFORT_BLOCK_SIZE.

BUGFIX/OPTIMIZATION: Lookup records from back.
@domcharrier
bugfix: tr*_dir*.py.in: Add missing arg to intrnl fun
1acbb93 
* add `scope` arg to signature of _intrnl_inout_arrays_in_subtree
* Fixes all tests in <gpufort_dir>/python/test/grammar_translator/openacc
@domcharrier
bugfix: linemapper: detect additional line cont. expr
d23cd4f 
* Detects now (additionally) expressions such as
  ```
  <line 1>&
  !$acc <rest of line 2>
  ```

  and removes the &\s*\n\s*!$acc
@domcharrier
BUGFIX/translator: Fix syntax error
328af33
@domcharrier
Add gpufort & linemapper feature
78825fb 
FEATURE/linemapper:
  Allow to prepend and append lines directly to
  statement data structures and not only whole line.
  New data structure triggered changes in all dependent packages (scanner,indexer)

FEATURE/gpufort:
  Add option to dump linemapper datastructure
@domcharrier
examples/openacc/vector-add-declare: Rename example
ab35838
@domcharrier
gpufort_acc_runtime: Add opt arg to indicate module variables
c5e528c
@domcharrier
scoper/search_*_for_*: Allow to turn error/warning off
527f587
@domcharrier domcharrier added the enhancement New feature or request label Oct 28, 2021
@domcharrier
BUGFIX/gpufort_acc_runtime: gpufort_acc_present_... dummy arg order
daf779c 
*Fix mismatching arg lists between wrapper/impl
function; put long dummy arg list of gpufort_acc_present_...
in macro and reuse macro in wrapper
(gpufort_acc_runtime) and implementation
(gpufort_acc_runtime_base).

minor/unrelated:
*rename internal function in gpufort.py (parse_cl_args)
@domcharrier
REFACTOR/gpufort_acc_rt: Combine 4 opt args into enum arg
ad93815
@domcharrier
vector-add-non-interop*: Showcase nested struct init and access.
e21f7fe
@domcharrier
Add some developer comments
7e67cba
@domcharrier
Readd gpufort_array/vector-add example
42c692b
@domcharrier
gpufort_array.h: Add __host__ print host/device data routine
3822779
@domcharrier
exa*/gpufort*/non-interop*-types: Change 1 original member to pointer
1005738
@domcharrier
test/gpufort_array/*: Change file names as expected by Makefile
fb9e5e4
@domcharrier
studies/gf*/interop*: Add c_funptr callback example
42077ae
@domcharrier
Add previously missing file
dc31f29
@domcharrier
gpufort_array: Introduce explicit async routines
513ef89 
* Refactor init/wrap/destroy/copy_to* routines to have blocking
  memcpy operations; remove stream argument.
* Introduce additional init_async/wrap_async/destroy_async/copy_to*_async
  routines with non-blocking memcpy operations.
@domcharrier
exam*/gpufort*/vector-add-interop*: Add example showcasing init of in…
a10983b 
…terop type
@domcharrier
test/linemapper: Add multiline directive & singline if tests.
8ca576a
@domcharrier domcharrier changed the title New OpenACC features for GPUFORT runtime WiP: New OpenACC features for GPUFORT runtime Nov 25, 2021
@domcharrier domcharrier self-assigned this Nov 25, 2021
This was linked to issues Nov 25, 2021
Line continuation character not always recognized #30
Closed
Inline if statements are not supported #31
Closed
domcharrier and others added 6 commits November 25, 2021 05:52
@domcharrier
Add previously missing test files.
0675bbf
@domcharrier
Merge branch 'develop-acc' of github.com:RocmSoftwarePlatform/gpufort…
7d218fb 
… into develop-acc
@domcharrier
WiP: Update test collection (test3-5 fail)
7d2c2ee 
TODO: Improve test parkour for translator
to improve declaration parsing.
@domcharrier
Merge branch 'develop-acc' of github.com:rocmsoftwareplatform/gpufort…
3a0cad8 
… into develop-acc
@domcharrier
Add slides on GPUFORT arrays
bbded71
@domcharrier
BUGFIX:translator: logical expr and => expr in decl RHS
ef79dce 
* Fix issue with parsing expressions that have '=>' in declared variable
  RHS.
* RHS of declared variable can now be logical expression too.
* Add more rigorous test for declaration.
This was linked to issues Dec 2, 2021
Failure to parse logical parameter statement #32
Closed
Failure to parse null pointer assignment in declaration #33
Closed
@domcharrier
WiP: Try to support comments in multi-line statements
ab67887 
GPUFORT tries to preserve comments.
Unfortunately, this becomes
difficult when a comment begins
after a line continuation character.

GPUFORT will move these comments
to the before the statement that
contains them.
@domcharrier
WiP
4ed0827
@domcharrier
Finalise inline comment reloc
4ea4fc6 
* Add test to folder python/test/utils
@domcharrier
Add index entry for CUDA builtin variables
f79e47d
@domcharrier
Merge pull request #37 from ROCmSoftwarePlatform/bugfix/comments-in-m…
b9186fc 
…l-statement

WiP: BUGFIX: Support comments in multi-line statements
@domcharrier
linemapper: Put inline comments behind statements
d319b1c
@domcharrier
BUGFIX/parsingutils: Check string length before checking char
ad5d85b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@domcharrier domcharrier

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

3 participants

@domcharrier @sael9740