Specific, specialty domains often have languages in which lots of code already exists. I did a bit of scientific programming in FORTRAN. I did actuarial programming in APL (ARIMA models for fun and profit). However, unless there's a LOT of conveniently available, pre-existing programs and good support for new development in FORTRAN in your current environment, I recommend against FORTRAN.
If you want good solid, mainstream language well-suited to mathematical and scientific tasks pick C. It has wide support, good, free compilers and development tools. There's a ton of existing, Open Source libraries for doing ALL manner of specialty tasks. C has the virtue that, if you decide to move forward (chronologically, historically speaking) to an object oriented paradigm, C++ is a very logical evolution, and you'll already know a great deal about the purely procedural aspects, datatypes, and control structures.
The book from which I learned C is old and presumes a solid basic grasp of programming principles - would not be good primer. There is a site that claims to offer some free texts at:
http://www.computer-books.us/c.php
Can't vouch the quality, but did I mention "free"?