4.5 extract.txt

The /etc/mediatex/mdtx-COLL/extractXXX.txt files handle the extraction rules for a collection.

Using the “$ mediatex upload[+] [file file [as target]]* [catalog file] [rules file] to coll coll” query, PUBLISHER (see Who) should provide extraction rules to the new archive to upload, using the same grammar language.

Relational schema:

• Proper entities
  • The CONTAINER entity lists (using an AVL tree) all the container files from where archives are extracted. A Container is identified by its type (ISO,TAR...) and the first archive needed for extraction (ex: archive-part1.rar), also called its first parent. Containers may provide a relative directory path to use when extracting the archive. This allow for instance to extract TAR container, that handle files on the root, into a directory. Notice: 2 dedicated CONTAINER instances are used internally by MEDIATEX (see Conceptual model) to manage the incoming (recently uploaded) archives, and where to extract supports into the cache.
  • The LOCATION entity lists external ressources (directories). MEDIATEX offers facility to serve files located into a local directory, for instance because theses files are already managed by another software and still may evolve. Locations are not cover by the scoring and geographical duplication algorithms like containers.
• Linked entities
  • The ARCHIVE entity lists the archive’s data files
• Relations
  • The FROMASSO relationship enumerate (using an AVL tree) the content of a container. For instance, all files provided by a tarball archive: tar -tf archive.tar. Notice: for the incoming CONTAINER instance, the FROMASSO relationship gives the date the archive was uploaded.
  • The PARENTS relationship gives all the parts a container is using to perform extraction. (archive-part1.rar, …, archive-partN.rar).
  • The TOCONTAINER relationship gives the container object an Archive may provides.

    Notice: we can only have on containers for an archive: in the case of a compressed tarball for instance, we have to choose between the two possibilities (TAR+GZ or TGZ).

  • The FROMLOCATION relationship gives files accessible by the collection user from the location directory. These “located files” are not true archives ; they still may be uploaded as “archives” later.

Exemple:

(INC
=>
0a7ecd447ef2acb3b5c6e4c550e6636f:374784	1994-01-01,00:00:00
de5008799752552b7963a2670dc5eb18:391168	2010-01-01,01:00:00
)

(IMG
=>
3a04277dd1f43740a5fe17fd0ae9a5aa:24457  here/logo.gz
)

(ISO
de5008799752552b7963a2670dc5eb18:391168
=>
1a167d608e76a6a4a8b16d168580873c:20480 logoP1.cat
)

(ISO
0a7ecd447ef2acb3b5c6e4c550e6636f:374784
=>
c0c055a0829982bd646e2fafff01aaa6:4066  logoP2.cat
)

(CAT
1a167d608e76a6a4a8b16d168580873c:20480,
c0c055a0829982bd646e2fafff01aaa6:4066
=>
0387eee9820fa224525ff8b2e0dfa9be:24546  logo.tgz
)

(TGZ
0387eee9820fa224525ff8b2e0dfa9be:24546
=>
directory: here
022a34b2f9b893fba5774237e1aa80ea:24075  logo/logo.png,
b281449c229bcc4a3556cdcc0d3ebcec:815    logo/logo.xpm
)

(COPY
746d6ceeb76e05cfa2dea92a1c5753cd /location
=>
022a34b2f9b893fba5774237e1aa80ea:24075	logo.png
b34bb9bf9ae4ec5b4a5bc2ab3e2a18c5:25088	logo.cpio
)

Notice:

• The INC container is used to remind the date new incoming where uploaded in order to let a delay, uploadTTL /etc/mediatex/mdtx-COLL/servers.txt for PUBLISHER before the uploaded file’s score impact the whole collection score.
• The IMG container is used to remind an extraction path to used when a support need to be copied into the cache. This feature is optional ; else the supports/ dirname concatenated with the support’s name/basename will be used.

Grammar:

file: stanzas
    | //empty file

stanzas: stanzas stanza
       | stanza

stanza: ( container => options childs )
      | ( location =>  options childs )

container: orphaneContainer
         | stdContainer
         | stdContainer parents

orphaneContainer: type 

stdContainer: type archive

options: DIRECTORY: string
       | /* none */

location: type hash string string
        | type hash string

parents: parents parent
       | parent

childs: childs child
      | child

parent: archive 

child: archive string

archive: hash : size

hash: [[:xdigit:]]{32}
string: [^[:blank:]\)\n\r]{1,511}
type: INC | IMG | ISO | CAT | TGZ | TBZ | TAR | CPIO
    | GZIP | BZIP | ZIP | RAR | LOCATION
size: [[:digit:]]+

Code:

src/memory/extractTree.h

src/memory/extractTree.c

src/parser/extractFile.l

src/parser/extractFile.y