9
$\begingroup$

Dear earth science community,

I am seeking a language to modernize a large Fortran project that works with geospatial NetCDF files. The requirements for this language include:

  • A rock-solid NetCDF library (we do not want to reinvent the wheel).
  • Excellent support for vectors and matrices (e.g. via operator overloading or something similar).
  • Native performance is needed (compiled or JIT-compiled).
  • Must be open-source.
  • Ideally, simple bindings with Fortran (not a strict requirement).

Python +numpy isn't really an option because of customized number-crunching with native performance. Even if numpy is implemented in C, the project includes custom algorithms that operate on matrices, and we do not want to write those in C. I know that Python could be optimized by calling into C or Fortran, but I just don't think that such optimization is going to happen with our team. Therefore, I am seeking a language that is "fast by default". Julia seems to deliver much better on this "fast by default" promise, whereas Python is more like "fast if you have the right optimizations in place".

Matlab is a non-starter because of its proprietary license.

C is a non-starter because it does almost nothing to "modernize" a Fortran codebase.

Go is probably not sufficiently well-established for scientific computing?

Julia could be a promising option because it combines the ease of Python with native performance.

Kotlin might be interesting because it combines the strength of the JVM with the possibility to compile to native code.

Swift would be a great language, but I am not sure whether it is sufficiently mature for my targeted domain.

C++20 and Rust are additional options that might be interesting, but I fear the complexity of those languages (remember that this is for earth scientists, not computer scientists).

Which options would you prefer for a gradual Fortran replacement with NetCDF files?

Improve this question
$\endgroup$
13
  • $\begingroup$ I would give a try to rust. But I think, it is very likely that you will end up with python, independently what advices you get here. $\endgroup$
    – peterh
    Nov 12, 2020 at 20:54
  • $\begingroup$ While I agree that Python has a great ecosystem, my group isn't really Python-fanatic. In fact, one of our main requirements is to run fast with custom matrix manipulations, and Python isn't that great for this. $\endgroup$
    – Mike76
    Nov 12, 2020 at 20:58
  • $\begingroup$ I think your group is right. There is a python library named tensorflow which can utilize 3d accelerators for complex operations. In practice I have never seen it working correctly and faster as if it would be purely in python :-) In exchange, I think most python libraries are crap, and the language speficially catalyzes the crap solutions. $\endgroup$
    – peterh
    Nov 12, 2020 at 21:01
  • 1
    $\begingroup$ I'm curious what requirement you have that isn't satisfied by your current Fortran code, I can't really glean it from the question. (Asking because one day I might run into the same problem!) $\endgroup$
    – Deditos
    Nov 13, 2020 at 15:21
  • 1
    $\begingroup$ @Mike76: Fast iteration times and modern scripting languages are pretty much mutually exclusive. Plus Fortran is easy to understand, Python is not, at least IMHO. $\endgroup$
    – jamesqf
    Nov 19, 2020 at 4:11

4 Answers 4

Reset to default
11
$\begingroup$

The modern language you are looking for is called modern Fortran 2018. Fortran 2008 and 2018 have everything a numerical computing project would need and so many features that many other languages mentioned here lack (including extremely pleasant array-syntax which has inspired all other languages such as MATLAB, Python, R, Julia, ..., even C++ numerical libraries). It is the only language that has native built-in parallelism and is the only language, along with C, for which the official MPI parallelism standard is released. It has also excellent vendor support for GPU parallel computing and as I heard in SC18, some vendors like NVIDIA/PGI have already started implementing the native parallelism features of Fortran via GPUs.

Modern Fortran is a high-level language comparable to MATLAB and Python, yet 100-500 times faster than both (https://modelingguru.nasa.gov/docs/DOC-2783). Using Coarray Fortran you can seamlessly parallelize your code with minimal effort to run it from your laptop to the largest supercomputers in the world. No other language has such capability at the moment. Fortran is also the only language that has all levels of parallelism, from instruction level to vectorized, concurrent, and coarray-distributed parallel computing. If such features are not considered "modern", perhaps "post-modern" would be a better description.

Fortran has been a reliable lasting language for almost 3 quarters of a century and has excellent highly-optimized compiler support (Intel, GNU, NAG, IBM, PGI/NVIDIA, ...). If you want to write code that lasts decades, then port your FORTRAN77 code to modern Fortran with minimal effort and save yourself time, energy, money, and computational power.

A good start with modern parallel object-oriented Fortran is "Modern Fortran Explained: Incorporating Fortran 2018" by Metcalf et al.

If you decide to port your code effortlessly to modern Fortran, you can also get help from the official Fortran language website https://fortran-lang.org/ and the community of programmers on Fortran discourse: https://fortran-lang.discourse.group/

Update Nov 17, 2020

Just yesterday NVIDIA released a news post on its GPU implementation of the concurrent features of Fortran 2008 / 2018. Users can now write code in pure modern Fortran and let the compiler automatically offload it to GPUs for thousands-way parallelism.

What is the relevance and importance of this achievement? The same Fortran code that a user writes can now be parallelized via GPUs, distributed CPUs, OpenMP threads, or auto-parallelization features of the Fortran compiler depending on the user's platform and compiler choice. All of that complexity is hidden from the user. No need for preprocessing and conditional compilations anymore. This further enhances the excellent existing portability of Fortran programs across platforms and architectures.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for sharing, what a shame that I didn't think about this. My current language is Fortran 90. $\endgroup$
    – Mike76
    Nov 14, 2020 at 2:59
  • $\begingroup$ I was in your position 5 years ago, when starting a new large-scale scientific project (almost) from scratch. We had this discussion of choosing between Fortran and C++. The final decision was made in favor of Fortran (2008). and the results have been quite satisfying so far. Its nice powerful syntax truly simplified life for the developers in our team. That said, do not go with any standard below 2003, if you choose Fortran. Fortran 2003 and beyond provide many powerful concepts that make Fortran 90 outdated. $\endgroup$
    – Scientist
    Nov 14, 2020 at 3:04
  • 1
    $\begingroup$ Thank you for directing me back into the right direction. When asking this question, I was ranting about the bad string support of Fortran 90. But then I looked at our own codebase and realized that there is almost nothing except of vectors and matrices... $\endgroup$
    – Mike76
    Nov 14, 2020 at 3:19
  • 3
    $\begingroup$ Almost any string operation that you may need is already implemented by the Fortran community. Also, Fortran 2003 and beyond have allocatable strings that are high-level and feel much like Python and MATLAB. For example this code character(:), allocatable :: string; string = "Fortran"; write(*,*) string; string = string // " 2018"; write(*,*) string; end is completely valid. Try it here: tutorialspoint.com/compile_fortran_online.php $\endgroup$
    – Scientist
    Nov 14, 2020 at 3:45
  • 1
    $\begingroup$ The Intel Fortran forum and Fortran Reddit community are also great for seeking help on Fortran, in particular, the most recent standards. $\endgroup$
    – Scientist
    Nov 14, 2020 at 3:49
7
$\begingroup$

As a former Fortran programmer I did a small online review several weeks ago into current trends for scientific programming.

To begin with, despite its age and its sometimes archaic style of programming, because of its huge legacy, Fortran isn't being directed to the trash can in a hurry. It's going to be used for a long time to come.

Like you, others have been looking for an alternative to start the move away from Fortran.

Though very popular, Python is very slow when executing programs. The speed of Fortran is another reason why it is still being used.

For astronomical and physics applications, I've seen a number of references state a preference for initially teaching students Fortran but when they get more experienced in their field exposing them to C++.

This comment from 2013, from a Physics forum encapsulates the general ideas I seen:

The older guys use fortran and c++. The younger people use python,java,and c++.

As with your answer, I think if Julia was more widely used it could have the potential to replace Fortran and possibly Python for scientific computing.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Thanks for sharing. I know that Python could be optimized by calling into C or Fortran, but I just don't think that such optimization is going to happen with our team. Therefore, I am seeking a language that is fast by default. Julia seems to deliver much better on this "fast by default" promise, whereas Python is more like "fast if you have the right optimizations in place". $\endgroup$
    – Mike76
    Nov 12, 2020 at 23:06
  • 1
    $\begingroup$ For our group, having a gigantic language-community isn't that important, as long as it is a well-maintained language. All we need is NetCDF and basic matrix-operations, so we could give Julia a try even if it isn't as mature as Python. $\endgroup$
    – Mike76
    Nov 12, 2020 at 23:09
  • 1
    $\begingroup$ @Mike76: If a lot of number crunching is involved, speed of processing is important. From reports of speed bench mark studies I've seen on line, in order of speed, the fastest 3 languages generally are C, C++ & Fortran. Julia is usually in the next tier of languages at positions 4, 5 or 6. I'd recommend Julia. $\endgroup$
    – Fred
    Nov 12, 2020 at 23:16
  • 1
    $\begingroup$ From what I have seen, Julia is "close to native performance". This seems to be good enough; I am willing to sacrifice a small bit of performance in order to overcome the slow development cycles of C/C++/Fortran. $\endgroup$
    – Mike76
    Nov 12, 2020 at 23:20
1
$\begingroup$

Right now, I think that Julia makes the cut, because it provides a great tradeoff between "fast scripting" and "high-performance scientific computing". But I am willing to hear other people's opinions, perhaps I have overlooked something.

Improve this answer
$\endgroup$
0
$\begingroup$

You could try Haskell, a modern functional language. It's straight forward to use but perhaps its lack of familiarity may a little intimidating for the uninitiated. It now comes with an extensive library.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.