Suggestions on mimicing fgetc and fseek with F90  
Author Message
m sutton





PostPosted: 2005-10-26 9:30:49 Top

fortran, Suggestions on mimicing fgetc and fseek with F90 Hi all,

I know that many compilers have hooks to the C routines fgetc and fseek.
I' However, I was hoping to use standard F90 I/O to do this. I
read data files that vary from 3MB to 45MB in size. So sometime the
whole file is read to access data. Other times I seek into the file to
pluck a few bytes out.

Would something like the following be practical

To mimic reading a file in a fgetc fashion

character(1) c1
character(100) text100
open(iunit, fname='foo', form='formatted',access='sequential')

do i =1,100
read(iunit,'(A1)',advance='no')c1
text100(i:i) = c1
end do


The to do a seek followed by a read I was thinking of something like the
following that would read characters starting at the 300th character and
stopping at the 399th character.

character(1) c1
character(100) text100
open(iunit, fname='foo', form='formatted',access='direct',recl=1)

j = 1
do i =300,399
read(iunit,rec=i)c1
text100(j:j) = c1
j = j +1
end do


Does this seem like a reasonable approach? Are there any known speed
issues with this approach? Fgetc and fseek are working for me now, but
I would like to removed the non-standard fortran calls.

Thanks in advance,

Mike Sutton

 
nospam





PostPosted: 2005-10-26 11:29:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 m sutton <email***@***.com> wrote:

> To mimic reading a file in a fgetc fashion
...
> do i =1,100
> read(iunit,'(A1)',advance='no')c1
> text100(i:i) = c1
> end do

Perhaps I misrecall (as I don't tend to do things like this, so I'd have
to check to be sure), but I don't think this will ever read past the end
of the first record.

> The to do a seek followed by a read I was thinking of something like the
> following that would read characters starting at the 300th character and
> stopping at the 399th character.
>
> character(1) c1
> character(100) text100
> open(iunit, fname='foo', form='formatted',access='direct',recl=1)
>
> j = 1
> do i =300,399
> read(iunit,rec=i)c1
> text100(j:j) = c1
> j = j +1
> end do
>
> Does this seem like a reasonable approach? Are there any known speed
> issues with this approach? Fgetc and fseek are working for me now, but
> I would like to removed the non-standard fortran calls.

Many, many issues. Start with the fact that you aren't guaranteed to be
able to open an arbitrary file as direct access at all, though you
probably can on most current systems. Then there is the problem that,
even if you can do such an open, the conversion between the resulting
"record numbers" and file position is not well-defined and portable. In
particular, you are likely to find "surprises" in switching between
unix-like systems (where end-of-record for text files tends to be a
single byte), vs dos/windows-like systems where it tends to be two bytes
(and we'll ignore vms-like systems where things are more different than
that).

Oh, and your read is unformatted, while the open is formatted. I'll
assume that is just a typo; in any case it is easily fixed. You can
concievably do character data either way, but you do need to be
consistent, and each has a slightly different set of subtle portavility
issues.

Yes, reading character-at-a-time can have speed issues. So can direct
access in general (it can turn off some read-ahead optimizations). They
might not be significant enough to worry about for your application;
that part is hard to tell. If you are doing formatted I/O, then pretty
much by definition, you aren't going to be going blazingly fast anyway,
the format processing might dwarf anything else, and you probably aren't
dealing with gigabyte-sized files.

All those "issues" aside, it might work adequately for you. Things
aren't as bad as, on rereading the above, I've made it sound. Might even
be the best current approach for your application, all things
considered. But don't think that this will make your code perfectly
standard-conforming, portable, and bullet-proof.

Come f2003 compilers, I think you'll want to look at stream I/O, but the
day for that is not quite yet here.

--
Richard Maine | Good judgement comes from experience;
email: last name at domain . net | experience comes from bad judgement.
domain: summertriangle | -- Mark Twain
 
Clive Page





PostPosted: 2005-10-28 1:57:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 In message <1h50833.ighhr11xx8akyN%email***@***.com>, Richard Maine
<email***@***.com> writes
>Come f2003 compilers, I think you'll want to look at stream I/O, but the
>day for that is not quite yet here.
>

Richard has perhaps forgotten (or chosen to ignore) the fact that the
g95 compiler (which you can download free from www.g95.org) implements
the F2003 Stream I/O specification in all its glory. I wrote some notes
on how it works at here:
http://www.star.le.ac.uk/~cgp/streamIO.html



--
Clive Page
 
 
kargl





PostPosted: 2005-10-28 2:16:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 In article <email***@***.com>,
Clive Page <email***@***.com> writes:
> In message <1h50833.ighhr11xx8akyN%email***@***.com>, Richard Maine
> <email***@***.com> writes
>>Come f2003 compilers, I think you'll want to look at stream I/O, but the
>>day for that is not quite yet here.
>>
>
> Richard has perhaps forgotten (or chosen to ignore) the fact that the
> g95 compiler (which you can download free from www.g95.org) implements
> the F2003 Stream I/O specification in all its glory. I wrote some notes
> on how it works at here:
> http://www.star.le.ac.uk/~cgp/streamIO.html

Perhaps, Richard noticed that the OP specifically requested a
"standard F90 I/O to do this".

--
Steve
http://troutmask.apl.washington.edu/~kargl/
 
 
nospam





PostPosted: 2005-10-28 2:21:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 Clive Page <email***@***.com> wrote:

> In message <1h50833.ighhr11xx8akyN%email***@***.com>, Richard Maine
> <email***@***.com> writes
> >Come f2003 compilers, I think you'll want to look at stream I/O, but the
> >day for that is not quite yet here.
>
> Richard has perhaps forgotten (or chosen to ignore) the fact that the
> g95 compiler (which you can download free from www.g95.org) implements
> the F2003 Stream I/O specification in all its glory.

A little bit of both (forgotten and chosen to ignore). I had forgotten,
though your mention reminds me that I did see mention of this feature
being done. I was "ignoring" it in that the poster seemed to have
interest in portability, and having the feature in one or two compilers
doesn't yet get you there.

Still, your point is well taken. And g95 is running on enough different
platforms that you do have at least a certain kind of portability with
it today (with more in the future as other compilers implement this).

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain| experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain
 
 
Clive Page





PostPosted: 2005-10-28 5:55:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 In message <djr5hc$cra$email***@***.com>, Steven G. Kargl
<email***@***.com> writes
>Perhaps, Richard noticed that the OP specifically requested a
>"standard F90 I/O to do this".
>
Well maybe that's the explanation. Of course there really isn't such a
thing, as F90 didn't support it. There are three solutions as far as I
know:

(1) Use g95 which is a Standard F90 compiler supporting stream I/O
conforming to the latest Standard. So far this solution isn't very
portable because I don't know of other compilers which have added this
F2003 feature, but no doubt more will be along soon. But it seems to me
to work well, and is the most future-proof.

(2) Use some other Fortran90 compiler which has stream or binary I/O as
a proprietary extension. Not, in my opinion, as good as (1).

(3) Do it using formatted direct-access with RECL=1, which works with
many compilers but is awkward to use (though it can be wrapped in a set
of subroutines), inefficient (I suspect), and not totally portable.




--
Clive Page
 
 
Paul Van Delst





PostPosted: 2005-10-28 6:25:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 Clive Page wrote:
> In message <djr5hc$cra$email***@***.com>, Steven G. Kargl
> <email***@***.com> writes
>
>> Perhaps, Richard noticed that the OP specifically requested a
>> "standard F90 I/O to do this".
>>
> Well maybe that's the explanation. Of course there really isn't such a
> thing, as F90 didn't support it. There are three solutions as far as I
> know:
>
> (1) Use g95 which is a Standard F90 compiler supporting stream I/O
> conforming to the latest Standard. So far this solution isn't very
> portable because I don't know of other compilers which have added this
> F2003 feature, but no doubt more will be along soon. But it seems to me
> to work well, and is the most future-proof.

I reckon (1) is the way to go .... along with a request to one's compiler vendor (assuming
one also has a licensed compiler along with g95) to start adding f2003 features. I mean,
that's what upgrade/maintenance fees should be for, no? :o) I would think the addition
of the f2003 access="stream" feature on I/O would be one of the more easier additions to
make to current compilers (seeing as most of them have it as extensions under a variety of
different names) if it hasn't been done already.

<aside>
Apart from allocatable components of derived types, one of the other things I'm hanging
out for in f2003 is the PROTECTED attribute for module variables (I wouldn't think that
would be difficult to implement either, but it's probably the total opposite).
</aside>

Questions from managers can be answered with your future-proofing argument. It might also
stimulate a bit more groundswell for faster development of, and transition to, f2003
compilers.

Anyway... it's getting late. ehem.

cheers,

paulv

--
Paul van Delst
CIMSS @ NOAA/NCEP/EMC
 
 
James Giles





PostPosted: 2005-10-28 7:11:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 Clive Page wrote:
> In message <djr5hc$cra$email***@***.com>, Steven G. Kargl
> <email***@***.com> writes
>> Perhaps, Richard noticed that the OP specifically requested a
>> "standard F90 I/O to do this".
>>
> Well maybe that's the explanation. Of course there really isn't
> such a thing, as F90 didn't support it. [...]

Well, all of the functionality of *formatted* streams was
already supported in F90. The F2003 formatted stream
feature adds nothing to the capability of the Fortran
language.

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare


 
 
m sutton





PostPosted: 2005-11-2 11:32:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 Thanks for the ideas. For now I will stick with fgetc & fseek even
though they are not standard. I have to support the Sun, CVF, PGI and
SGI compilers. These are the compilers that the users have.

I have not tried g95 or gfortran yet. It looks like gfortran does not
support fgetc and fseek.

Mike Sutton
 
 
kargl





PostPosted: 2005-11-3 2:02:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 In article <orW9f.5760$email***@***.com>,
m sutton <email***@***.com> writes:
>
> I have not tried g95 or gfortran yet. It looks like gfortran does not
> support fgetc and fseek.
>

It's on the gfortran TODO list.

--
Steve
http://troutmask.apl.washington.edu/~kargl/
 
 
robert.corbett





PostPosted: 2005-11-3 17:43:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 > Apart from allocatable components of derived types, one of the other things I'm hanging
> out for in f2003 is the PROTECTED attribute for module variables (I wouldn't think that
> would be difficult to implement either, but it's probably the total opposite).

Sun f95 has supported the PROTECTED attribute for more than three years
now.
It was not difficult to implement. It did require extensions to Sun's
module file
format, and they were not done correctly; but that was a matter of
process, not
technology.

Bob Corbett

 
 
robert.corbett





PostPosted: 2005-11-3 17:45:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 > (1) Use g95 which is a Standard F90 compiler supporting stream I/O
> conforming to the latest Standard. So far this solution isn't very
> portable because I don't know of other compilers which have added this
> F2003 feature, but no doubt more will be along soon.

Sun f95 has had stream access for more than three years now.

Bob Corbett

 
 
Dick Hendrickson





PostPosted: 2005-11-4 1:23:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90

email***@***.com wrote:
>> Apart from allocatable components of derived types, one of the other things I'm hanging
>>out for in f2003 is the PROTECTED attribute for module variables (I wouldn't think that
>>would be difficult to implement either, but it's probably the total opposite).
>
>
> Sun f95 has supported the PROTECTED attribute for more than three years
> now.
> It was not difficult to implement. It did require extensions to Sun's
> module file
> format, and they were not done correctly; but that was a matter of
> process, not
> technology.
>
> Bob Corbett
>
There's a potential gotcha in PROTECTED variables. The
standard basically says they can't be defined (or undefined)
outside of the module. That's easy enough for the compiler
to enforce in the USEing routine, but not if they are
passed on to an external subroutine.
use some_module, only : a_protected_variable
call some_random_external (a_protected_variable)
and you're on your own. There's no guaranteed compiler
help with the protection.

Dick Hendrickson

 
 
Paul Van Delst





PostPosted: 2005-11-4 1:34:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 Dick Hendrickson wrote:
>
>
> email***@***.com wrote:
>
>>> Apart from allocatable components of derived types, one of the other
>>> things I'm hanging
>>> out for in f2003 is the PROTECTED attribute for module variables (I
>>> wouldn't think that
>>> would be difficult to implement either, but it's probably the total
>>> opposite).
>>
>>
>>
>> Sun f95 has supported the PROTECTED attribute for more than three years
>> now.
>> It was not difficult to implement. It did require extensions to Sun's
>> module file
>> format, and they were not done correctly; but that was a matter of
>> process, not
>> technology.
>>
>> Bob Corbett
>>
> There's a potential gotcha in PROTECTED variables. The
> standard basically says they can't be defined (or undefined)
> outside of the module. That's easy enough for the compiler
> to enforce in the USEing routine, but not if they are
> passed on to an external subroutine.
> use some_module, only : a_protected_variable
> call some_random_external (a_protected_variable)
> and you're on your own. There's no guaranteed compiler
> help with the protection.

Hmm. How is this different from the case where one does:

call some_random_external(1)

subroutine some_random_external(j)
integer j
j = j+1
end subroutine some_random_external

?? (If at all)

If, in your
use some_module, only : a_protected_variable
call some_random_external (a_protected_variable)
example, the external routine modifies the protected variable, what happens upon return to
the calling routine? I.e what value does "a_protected_variable" contain?

cheers,

paulv

--
Paul van Delst
CIMSS @ NOAA/NCEP/EMC
 
 
Dick Hendrickson





PostPosted: 2005-11-4 1:45:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90

Paul Van Delst wrote:

> Dick Hendrickson wrote:
>
>>
>>
>> email***@***.com wrote:
>>
>>>> Apart from allocatable components of derived types, one of the other
>>>> things I'm hanging
>>>> out for in f2003 is the PROTECTED attribute for module variables (I
>>>> wouldn't think that
>>>> would be difficult to implement either, but it's probably the total
>>>> opposite).
>>>
>>>
>>>
>>>
>>> Sun f95 has supported the PROTECTED attribute for more than three years
>>> now.
>>> It was not difficult to implement. It did require extensions to Sun's
>>> module file
>>> format, and they were not done correctly; but that was a matter of
>>> process, not
>>> technology.
>>>
>>> Bob Corbett
>>>
>> There's a potential gotcha in PROTECTED variables. The
>> standard basically says they can't be defined (or undefined)
>> outside of the module. That's easy enough for the compiler
>> to enforce in the USEing routine, but not if they are
>> passed on to an external subroutine.
>> use some_module, only : a_protected_variable
>> call some_random_external (a_protected_variable)
>> and you're on your own. There's no guaranteed compiler
>> help with the protection.
>
>
> Hmm. How is this different from the case where one does:
>
> call some_random_external(1)
>
> subroutine some_random_external(j)
> integer j
> j = j+1
> end subroutine some_random_external
>
> ?? (If at all)
I don't think it's fundamentally different. It's a
programming error, and one which compilers generally
have trouble diagnosing.

>
> If, in your
> use some_module, only : a_protected_variable
> call some_random_external (a_protected_variable)
> example, the external routine modifies the protected variable, what
> happens upon return to the calling routine? I.e what value does
> "a_protected_variable" contain?

Unless I missed something in the standard, it's undefined.
The rule is ultimately a boy scout one, "Thou shalt not mess
up a protected variable". If you do, your program is non
standard and anything can happen.

Protected variables might be stored in read-only memory,
or they might be passed-by-value, but you can't count it.

Dick Hendrickson
>
> cheers,
>
> paulv
>

 
 
James Giles





PostPosted: 2005-11-4 1:54:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 Paul Van Delst wrote:
> Dick Hendrickson wrote:
...
>> There's a potential gotcha in PROTECTED variables. The
>> standard basically says they can't be defined (or undefined)
>> outside of the module. That's easy enough for the compiler
>> to enforce in the USEing routine, but not if they are
>> passed on to an external subroutine.
>> use some_module, only : a_protected_variable
>> call some_random_external (a_protected_variable)
>> and you're on your own. There's no guaranteed compiler
>> help with the protection.
>
> Hmm. How is this different from the case where one does:
>
> call some_random_external(1)
>
> subroutine some_random_external(j)
> integer j
> j = j+1
> end subroutine some_random_external
>
> ?? (If at all)
>
> If, in your
> use some_module, only : a_protected_variable
> call some_random_external (a_protected_variable)
> example, the external routine modifies the protected variable, what
> happens upon return to the calling routine? I.e what value does
> "a_protected_variable" contain?

I think that's the point Dick was making. In both cases the
call is illegal. But, the standard doesn't require implementations
to detect or report the violation. And in fact, actual compilers
seldom will. It's not just a "quality of implementation" issue
since detecting this can actually cause significant degradation
in quality (specifically efficiency). It's one of those where
it's up to the individual programmer to be aware of the
problem and not do the illegal call.

(In your example above of passing a literal constant, I support
a change in the standard anyway. The rule should be that when
you pass literals or named constants the compiler should use
call-by-value semantics for that argument. That is, the procedure
may cause the dummy argument to be redefined or become
undefined but the caller ignores any change in value. That's
already a very common behavior of implementations when the
actual argument is an expression.)

--
J. Giles

"I conclude that there are two ways of constructing a software
design: One way is to make it so simple that there are obviously
no deficiencies and the other way is to make it so complicated
that there are no obvious deficiencies." -- C. A. R. Hoare


 
 
nospam





PostPosted: 2005-11-4 3:03:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 Dick Hendrickson <email***@***.com> wrote:

> Paul Van Delst wrote:
> > I.e what value does "a_protected_variable" contain?
>
> Unless I missed something in the standard, it's undefined.

I'll quibble with that answer. I think that what you have is illegal
code in that case. This in the category of illegal code for which
anything can happen. "Anything" encompasses a lot more than just the
value of the variable being undefined. For example, the code might well
abort wkith some kind of error. It might even fail to compile. You can't
count on the compiler detecting such things, but it is certainly allowed
to, and one could imagine scenarios in which it might be possible.

When the standard says that the value of a variable is undefined, that
at least ties down the effects to that variable. Illegal code doesn't
tie things down that much.

I'm sure that Dick knows all this stuff and was probably just speaking
loosely. My elaboration is just for other readers.

--
Richard Maine | Good judgment comes from experience;
email: my first.last at org.domain| experience comes from bad judgment.
org: nasa, domain: gov | -- Mark Twain
 
 
Jugoslav Dujic





PostPosted: 2005-11-4 15:47:00 Top

fortran >> Suggestions on mimicing fgetc and fseek with F90 Dick Hendrickson wrote:
| Paul Van Delst wrote:
|
|| Dick Hendrickson wrote:
||||
||| There's a potential gotcha in PROTECTED variables. The
||| standard basically says they can't be defined (or undefined)
||| outside of the module. That's easy enough for the compiler
||| to enforce in the USEing routine, but not if they are
||| passed on to an external subroutine.
||| use some_module, only : a_protected_variable
||| call some_random_external (a_protected_variable)
||| and you're on your own. There's no guaranteed compiler
||| help with the protection.
||
||
|| Hmm. How is this different from the case where one does:
||
|| call some_random_external(1)
||
|| subroutine some_random_external(j)
|| integer j
|| j = j+1
|| end subroutine some_random_external
||
|| ?? (If at all)
|
| I don't think it's fundamentally different. It's a
| programming error, and one which compilers generally
| have trouble diagnosing.
|
| Protected variables might be stored in read-only memory,
| or they might be passed-by-value, but you can't count it.

There is a likely difference in implementation though.
For constants (and expressions?), it's fairly easy for
compiler to put them in read-only memory (CVF and IVF
do that by default and you'll get a run-time error if
you try Paul's code).

However, for protected variables compiler can't do that
because it couldn't know when it will be (legally) written
by module procedures or (illegally) written by outside
procedures, and one can't mark OS memory pages as
"full access for these routines and read-only for others".

So, the bottom line is that it *is* a programmer's error
but I wouldn't be optimistic that compilers would be able
to catch it in all situations (but, for example, if dummy
argument j of some_random_external had an INTENT(INOUT)
declared & explicit interface provided, a compile-time
error would be justified).
--
Jugoslav
___________
www.xeffort.com

Please reply to the newsgroup.
You can find my real e-mail on my home page above.