English Translation Treebank: An-Nahar Newswire

Introduction

English Translation Treebank: An Nahar Newswire was developed by the Linguistic Data Consortium (LDC). It consists of 599 distinct newswire stories from the Lebanese publication An Nahar translated from Arabic to English and annotated for part-of-speech and syntactic structure.

This corpus is part of an ongoing effort at LDC to produce parallel Arabic and English treebanks. The files in this release are parallel with those in Arabic Treebank: Part 3 - v3.2 (LDC2010T08). Other parallel releases are (1) Arabic Treebank: Part 1 v 2.0 (LDC2003T06) (Agence France Presse newswire) and Arabic Treebank: Part 1 - 10K-word English Translation (LDC2003T07) and (2) Arabic Treebank: Part 1 v 3.0 (POS with full vocalization + syntactic analysis) (LDC2005T02) (Agence France Presse newswire) and its translated English counterpart English-Arabic Treebank v 1.0 (LDC2006T10).

The guidelines followed for both part-of-speech and syntactic annotation are Penn Treebank II style, with changes in the tokenization of hyphenated words, part-of-speech and tree changes necessitated by those tokenization changes and revisions to the syntactic annotation to comply with the updated annotation guidelines (including the Treebank-PropBank merge or Treebank IIa and treebank c changes).

Data

The data consists of 461,489 tokens in 599 individual files. The news stories in this release were published in An Nahar in 2002.

The English sources files (translated from the Arabic) were automatically tokenized, part-of-speech tagged and parsed the tokens, tags and parses were manually corrected. The quality control process consisted of a series of specific searches for over 100 types of potential inconsistency and parse or annotation error. Any errors found in those searches were manually corrected. The steps occurred in the following order:

• Automatic tokenization


• Human correction


• Automatic pre-tag and pre-parse


• Human annotation


• QC correction


• Automatic scripts for treebank c revisions

Annotations are in the following two formats:

• Penn Style Trees
  
  
  
  • Bracketed tree files following the basic form (NODE (TAG token)). Each sentence is surrounded by a pair of empty parentheses.
  
  
  
  


• AG xml
  
  
  
  • TreeEditor .xml stand-off annotation files. These files contain the POS and Treebank annotation and reference the source files by character offset. DTD files for the AG xml files were moved from their original location indicated in the readme to be more consistence with LDC publications.
  
  
  
  

Sponsorship

This work was sponsored in part by the Defense Advanced Research Projects Agency, GALE Program Grant No. HR0011-06-1-0003. The content of this publication does not necessarily reflect the position or policy of the Government, and no official endorsement should be inferred.

Samples

For a sample of this data consult the following links:

• AG XML


• Source SGML


• Penn Tree

Updates

None at this time.

Portions © 2002 An Nahar, © 2004-2012 Trustees of the University of Pennsylvania