14 library(hash_stream): Maintain a hash on a stream

See also

In addition to this hash library, SWI-Prolog provides library(md5), library(sha) and hash functions through library(crypto), part of the ssl package.

This library defines a filter stream that maintains a hash of the data that passes through the stream. It can be used to compute the hash of input data while it is being processed. This is notably interesting if data is processed from a socket as it avoids the need for collecting the data first in a temporary file.

A typical processing sequence is illustrated below, where process/2 somehow processed the data and save_result/3 records the result as obtained from URL with content digest SHA256 its Result.

    ...,
    http_open(URL, In0, []),
    open_hash_stream(In0, In, [algorithm(sha256)]),
    process(In, Result),
    stream_hash(In, SHA256),
    close(In),
    save_result(URL, SHA256, Result)

This library can also be used to compute the hash for the content of a file. The advantage is that this code doesn't rely on external tools. It is considerably faster for short files, but considerably slower on large files because Prolog I/O is based on character streams rather than blocks.

file_hash(Algorithm, File, Hash) :-
    setup_call_cleanup(
        open(File, read, In0, [type(binary)]),
        setup_call_cleanup(
            open_hash_stream(In0, In,
                             [ algorithm(Algorithm),
                               close_parent(false)
                             ]),
            ( setup_call_cleanup(
                  open_null_stream(Null),
                  copy_stream_data(In, Null),
                  close(Null)),
              stream_hash(In, Hash)
            ),
            close(In)),
        close(In0)).

[det]open_hash_stream(+OrgStream, -HashStream, +Options)

Open a filter stream on OrgStream that maintains a hash. The hash can be retrieved at any time using stream_hash/2. Provided options:

algorithm(+Algorithm)

One of md5, sha1, sha224, sha256, sha384 or sha512. Default is sha1.

close_parent(+Bool)

If true (default), closing the filter stream also closes the original (parent) stream.

[det]stream_hash(+HashStream, -Digest:atom)

Unify Digest with a hash for the bytes send to or read from HashStream. Note that the hash is computed on the stream buffers. If the stream is an output stream, it is first flushed and the Digest represents the hash at the current location. If the stream is an input stream the Digest represents the hash of the processed input including the already buffered data.