Input and Output

• In this section, we see how to do simple file handling with open().
• More often than not, you won't need to do that
  • For text files, use numpy.loadtxt() or numpy.genfromtxt
  • Otherwise, use dedicated libraries (h5py, fabio, ...)

Opening files

How to open a file ?

f = open(filename, mode='r') will create a file object f

mode can be:

• r for read-only mode
• w for write mode
• a for append mode
• + for read+write mode
• b for binary mode (disable encoding handling)

Common methods for all file objects:

• f.close() → close the file
• f.seek(pos) → Moves to a given position in the file

Writing into a file

• f.write(string) → write a string to the file
• f.writelines([list of strings]) → write multiple lines
• f.flush() → write (actually) the data to the disk

import time

filename = "example_file.txt"

f = open(filename, "w")
f.write("It is now: " + time.ctime())
f.flush()  # Optional
f.close()  # Mandatory

Reading from a file

• f.read(size) → read size bytes or the whole file in a string
• f.readlines() → read the whole file in a list of lines
• f.readline() → read the next line (by default defined by \n)

print("Current time:", time.ctime())
print("Time recorded in file:", open(fn1).read())

Nota: In the former cell, the file may not be closed immediately as the garbage collection can be delayed. As the number of opened files is limited at ~1000 per process, it is always better to explicitly close files.

This is best achieved using a context manager.

Context manager

• Context manager will allocate and release resources 'automatically' when needed.
• Usually used from the with statement.

To write safely into a file, instead of having something like:

file = open("my_file", "w")
try:
    file.write("Hi there!")
finally:
    file.close()

using a context manager will give:

with open("my_file", "w") as opened_file:
    opened_file.write("Hi there!")

The main advantage of using a with statement is to make sure your file will be closed (without dealing with the finally block in this case)

This will also increase code readability.

A common use case of context managers is locking and unlocking resources.

Yasoob Khalid Python tips regarding context manager:

http://book.pythontips.com/en/latest/context_managers.html

File as iterator

Files can behave as iterators over readlines

with open("data/dummy_file.txt") as f:
    for line in f:
        print(line)

will display:

first line
    second line
    ...

• Very concise typing
• Efficient reading
• Limited memory footprint (File is not fully loaded in memory → only one line at a time)

Hands on

Write into a file:

- your name
- the current date

Then, read back this file and parse it to retrieve the month of the date. Use only the functions given by the file object (the one returned by open(...))

Solution - writing

import time

with open("myoutputfile", mode="w") as opened_file:
    opened_file.write("Jerome\n")
    opened_file.write(time.ctime())

Solution - reading

# reading
with open("myoutputfile", mode="r") as f:
    # read the first line with the name
    firstline = f.readline()
    # read the second line with the date
    secondline = f.readline()
    month = secondline.split()[1]
print("month is %s" % month)

Hands on

A scattering geometry can be entirely described with a Point of Normal Incidence (PONI).
This is the coordinates of the orthogonal projection of center of sample onto the detector.

pyFAI uses a PONI file to describe the geometry.
It looks like this:

poni_version: 2.1
Detector: Eiger2_4M
Detector_config: {"orientation": 3}
Distance: 0.41826947072547654
Poni1: 0.1273645012076586
Poni2: 0.04721622971911041
Rot1: -0.006558391003187438
Rot2: -0.001319758222137916
Rot3: 4.987612019731912e-06
Wavelength: 1.6531226457760035e-11

Exercise read the above information from the file data/eiger.poni, and store it in a dictionary.
It should look like:

{'poni_version': '2.1',
 'Detector': 'Eiger2_4M',
 'Detector_config': ' {"orientation" 3}',
 'Distance': '0.41826947072547654',
 'Poni1': '0.1273645012076586',
 'Poni2': '0.04721622971911041',
 'Rot1': '-0.006558391003187438',
 'Rot2': '-0.001319758222137916',
 'Rot3': '4.987612019731912e-06',
 'Wavelength': '1.6531226457760035e-11'
}

Solution

def read_poni(filepath, comment="#"):
    "Read a PONI file"
    poni_info = {}
    with open(filepath, "r") as opened_file:
        for line in opened_file:
            # Sanitize current line: remove leading and trailing spaces
            line = line.strip()
            # Skip commented lines
            if line.startswith(comment):
                continue
            # Split the line before and after the ":" separator
            fields = line.split(":")
            name = fields[0]
            if len(fields) == 2:
                value = fields[1].strip()
            elif len(fields) > 2:
                value = "".join(fields[1:])
            else:
                continue
                value = value.strip()
            poni_info[name] = value
    return poni_info


data = read_poni("data/eiger.poni")

data

Interaction with the console

• Output to the console:
  • print(str)
• Input from the console:
  • input() reads the standard input and returns a string

user_name = input("Please enter your name:")
print("user name is", user_name)

Hands on

Create a function asking for the name and the age of the user and then display it

>>> What is your name ?
polo
>>> How old are you ? 
22
>>> Your name is polo and you are 22 years old

Solution

def questioneer():
    print("What is your name ?")
    name = input()
    print("How old are you ? ")
    age = input()
    print("Your name is %s and you are %s years old" % (name, age))


questioneer()

sys standard output, input and error

• sys.stdout, sys.stdin, sys.stderr:
  • File objects used for standard output, input and errors.
  • sys.stdout is a file open in write mode
  • sys.stdin is a file open in read mode
  • sys.stdout.write(str+os.linesep) is equivalent to print(str)

>>> import sys
>>> import os
>>> sys.stdout.write('Is winter coming ?' + os.linesep)
Is winter coming ?
>>> answer = sys.stdin.readline()
might
>>> print(answer)
might

example of stdin, stdout

import sys

sys.stdout = open("std_out_file", "w")
print("toto")
sys.stdout.flush()