RIMES, complete

IntroductionThe RIMES-database (Reconnaissance et Indexation de données Manuscrites et de fac similÉS / Recognition and Indexing of handwritten documents and faxes) comprises handwritten correspondence letters, in French, “sent” by individuals to companies or administrations; all correspondence is fictitious and there is no PII in the records. The database was collected by asking volunteers to write handwritten letters in exchange for gift vouchers. Volunteers were given a fictitious identity (same sex as the real one) and up to 5 scenarios. Each scenario was chosen from among 9 realistic topics: change of personal data (address, bank account), request for information, opening and closing (customer account), change of contract or order, complaint (e.g. poor quality of service), payment difficulties (request for delay, tax exemption, etc.), reminder, claim with other circumstances and a target (administrations or service providers such as telephone, electricity, bank, insurance companies). The volunteers wrote a letter with this information in their own words. The layout was free and the only request was to use white paper and to write legibly in black ink.ContentsThere are several files in this record; they correpond to the whole dataset and also some subsets that were created during the project. The Communications part is where the full document images and their annotations are stored. We give some detail of what the annotations comprise and briefly describe the other subsets; NB there was a subset comprising images from the cropped logos, which are not distributed here, due to some issues with the annotations.Communications  -- Images_Courriers.zipThere are in total 5605 communications, each containing from 2 to 3 pages:One correspondence (mandatory)One questionnaire (mandatory)One fax (optional)Filenames are constituted as follows:Communication number [1, 5605]Underscore “”One letter [F, L, Q]L for correspondence/Letter (Lettre, in French)QuestionnaireFaxThe images and the corresponding annotations are split into 3 folders:DVD1: images from 1 to 1799DVD2: images from 1800 to 3699DVD3: images from 3700 to 5605There are in total 12610 images.The annotation files are in xml format and support different tasks:Document Structure IdentificationHandwritten text recognitionWriter recognitionInformation ExtractionDocument Structure IdentificationFor the document structure, there are 8 "types" to be identified in the Letters for the different blocks of text in the image:Sender addressRecipient addressDate/Place (each is annotated with it's own tag)SubjectIntroduction (Ouverture in French)Text bodySignaturePS / annexTypes 1 and 2 have further details, if it is a person or a corporate entity and address can also contain telephone/fax number.For example:    Coordonnées Expéditeur  (Sender Address)  Maxime Granier\n13 Grand rue\n57370 Dames et Quatre Vents   In the case of Faxes, the types can be further complemented by the type of text:Dactylographié, which stands for Printed;Manuscrit, for Handwritten text.For instance:    Expéditeur_autre / Dactylographié  De :       Expéditeur_personne / Manuscrit  Lucie FOURES   Questionnaires have several other types, but there is no remanescent documentation about them. Handwritten text recognitionThere are annotations for the paragraphs; line breaks are indicated with "\n". The transcriptions are verbatim and contain the same spelling and grammar errors that could be seen in the pages. When there could be more than one possible spelling (j’essaie/j’essaye, événement/évènement, ultrason/ultra son, and writing errors), the options are in the ground truth following a special construction:  ¤{alternative_1/alternative_2}¤Writer recognitionThere is an identity for each writer in the database, so the usual tasks of identification and verification can be realized. The writer is identified in the usual French form, with family name, in all caps, first followed by the given name, for instance:GRANIER MaximeInformation ExtractionNine scenarios are annotated for the different types of communication. We provide some free translations into English.ScenarioFree translationChangement de données personnellesChange of personal dataDemande d'informationRequest for informationDifficulté de paiementPayment DifficultiesFermeture de compteAccount ClosureGestion de sinistreClaims HandlingModification de contrat / CommandeContract Changes / OrderOuverture de compteAccount OpeningRéclamationComplaintRelance de courrier sans réponseCorrespondance Reminder Paragraphs -- images_blocs_de_texte.zipThe main body of the letters was cropped from the full page images and stored as grayscale JPEG images and split into 3 folders:DVD1: images from 1 to 1799 (1796 images)DVD2: images from 1800 to 3699 (1899 images)DVD3: images from 3700 to 5605 (1905 images)The transcriptions can be obtained from the corresponding Communications transcriptions; the numbers in the filenames correspond to the communications.Cursive words -- imagettes_mots_cursif.zipThe paragraphs were split into lines and each line was further split into words.The images were split into 57 blocks (lot in French) organized in folders named:lot_N_rimes_version_definitive, where N is the block number [1, 57]Each folder has data from 100 letters, further organized into sub-folders following the convention:LThen each sub-folder has one image per word, with naming:L.tiffWhere Line Number and Word position start from 0. The transcription should be inferred from the corresponding Communications transcriptions.Character snippets -- imagettes_caracteres.zipWords were split to characters (A to Z) and digits (0 to 9), totalling 95269 images. They are distributed in 3 folders:characters_rimes_DVD1characters_rimes_DVD2characters_rimes_DVD3Each is further divided into 4 blocks (lot in French):lot_1lot_2lot_3lot_4The naming of the image files follows the following:_.pngWhere Class is in [A-Z0-9].Acknowledgments This dataset was originally collected and prepared in 2007 by the following partners: DGA/CTA/DT/GIP - CEP Arcueil; TSP – ARTEMIS Télécom SudParis; and A2iA SA, as part of the Techno-Vision project. This project was funded by the French ministries for Research and Defense (Ministère de la Recherche and Ministère de la Défense).After the acquisition of A2iA SA in September 2018,  Mitek Systems, Inc. became a legal owner of the dataset, and decided to release it publicly – which was one of the objectives of the project after its conclusion – under a permissive license in 2024, to encourage open science.