(Agreement Ref. No. LC/ELDA/EVAL/2007/000/NAME)





This agreement is made by and between:


".................................. ", (hereinafter called END-USER), having its principal place of business at: ..........................





ELDA S.A., (hereinafter called DISTRIBUTOR), the distribution agency commissioned by ELRA, having its principal place of business at: 55-57 rue Brillat Savarin - 75013 Paris, FRANCE

registered at the Tribunal de commerce de Paris:

RCS Paris B 402 781 876 (95b147 95)



whereby it is agreed as follows:


1.     Evaluation Packages, object of this Agreement, for which DISTRIBUTOR obtained distribution rights from the rightful holder, are described in Exhibit A.


2.     The site of use of the Evaluation Packages is mentioned in Exhibit B.


3.     Within this Agreement Distributor grants End-User, engaged in bona fide language engineering research, the non-exclusive right to use the Evaluation Packages, exclusively for the purposes of evaluating their Human Language Technologies.


4.     End-User is not permitted to reproduce the Evaluation Packages for commercial or distribution purposes and to commercialise (or distribute for free) in any form or by any means the Evaluation Packages or any derivative product or services based on all or a substantial part of it. Summaries, analyses and interpretations of the linguistic properties of the information may be derived and published, provided it is not possible to reconstruct the information from these summaries. Small excerpts of the information may be displayed to others or published in a scientific or technical context, solely for the purpose of describing the research and development and related issues. Any such use shall not infringe on the rights of any third party including, but limited to, the authors and publishers of the excerpts.


5.     Distributor grants End-User the right to reproduce the Evaluation Packages temporarily or permanently, to translate, adapt, arrange and modify by any means the Evaluation Packages, if one or all of these acts are necessary to access and make the agreed use of the contents of the Evaluation Packages. End-User is permitted to make a copy of the Evaluation Packages for purposes of archiving only.


6.     Without prejudice to the other provisions, the rights referred to herein shall be non transferable to any other entity. The Evaluation Packages shall not be transferred to or accessed from any other site.


7.     End-User acquires no ownership, rights or title in all or any parts of the Evaluation Packages.


8.     Distributor and Owners accept no responsibility for the accuracy or completeness of the data or for the consequences of their use. DISTRIBUTOR and Owners give no warranty for merchantibility and/or fitness for a particular purpose of the LRs.


9.     End-User shall give appropriate references to Distributor, as well as to the name and reference of the Evaluation Packages in scholarly literature when the Evaluation Packages are mentioned. The following acknowledgement is required: "ELRA catalogue (http://catalog.elra.info), EVALUATION PACKAGE NAME, catalogue reference: ELRA-XXXX."


10. End-User shall not use the name of Distributor in any publication in any manner that would imply an endorsement of End-User or any product or service offered by End-User.


11. END-USER has no right or authority to incur, assume or create, in writing or otherwise, any warranty, liability or other obligation of any kind, express or implied, in the name of or on behalf of DISTRIBUTOR, it being intended that each party shall remain an independent contractor responsible for its own actions.


12. Neither party shall be responsible for damages caused by the other party's non-fulfillment of provisions of this AGREEMENT.


13. Both parties exclude all liability of whatsoever nature for direct, consequential or indirect loss or damage suffered by the other.


14. END-USER agrees to pay DISTRIBUTOR a compensation. The mode of payment and schedule of payments are incorporated in Exhibit C and form part of this AGREEMENT.


15. In order to preserve the desired, pre- competitive nature of the Human Language Technology evaluations, DISTRIBUTOR has developed a set of guidelines constraining the dissemination and publication of evaluation results. These guidelines are meant to preclude the publication of incomplete or inaccurate information that could damage the reputation of DISTRIBUTOR or Human Language Technologies / players and could discourage participation in future evaluation campaigns. The guidelines (listed in Exhibit D) shall be implemented by END-USER.


16. This Agreement is subject to, construed and interpreted in accordance with the Law of France. Should it not be possible to settle amicably differences of interpretation out of this Agreement, then the case shall be brought before the regular courts of law for a decision. The "Tribunal de commerce de Paris" shall be the only competent court.



The entire Agreement is composed of the 16 articles herein together with Exhibits A, B, and C thereafter.


In witness whereof, intending to be bound, the parties hereto have executed this AGREEMENT by their duly authorized officers.







________________                                                                        ________________

On behalf of                                                                                       On behalf of ELDA

Name:                                                                                                 Name:  Khalid CHOUKRI

Title:                                                                                                    Title: Managing Director  

Date:                                                                                                    Date:






Exhibit A Evaluation Packages refer to:



Exhibit B SITE OF USE:


EXHIBIT C Compensation:


EXHIBIT D: Guidelines constraining the dissemination and publication of evaluation results are:


a)      SCIENTIFIC OR TECHNICAL PUBLICATIONS: Scientific or technical publications, including newsletters from universities or research laboratories, should adhere to community standards for fairness and objectivity and should accurately and clearly state the limitations of the testing conditions and other factors which might influence scores. The experimental nature of the tasks, data and evaluation procedures should also be stated. The full evaluation packages should always be referenced. 


b)      ADVERTISEMENTS: No advertisements using the evaluation results can be placed in magazines, journals, newspapers, or other publications. 


c)      PRESS RELEASES: Press releases about the evaluation results to organizations with national/international coverage are also prohibited. 


d)      MARKETING LITERATURE, LOCAL NEWSLETTERS: Although it is recognized that extensive evaluation discussions are not appropriate in this type of literature, it is expected that any claims made on the basis of evaluation results are accurate, that the evaluation measures used to substantiate these claims are stated, and that a reference is made to the evaluation packages. Where promotional material is subject to prepublication revision by the media, the author should make every effort to see that the revision does not cause a violation of the guidelines.


e)      CROSS-SYSTEM COMPARISONS: Cross-system comparisons may not be made with other named teams listed in the evaluation package documentation for individual tests, and may only be made when they are supported by accepted methods of statistical significance testing. Comparisons must be accompanied by the results of those tests and should reference the publication of those tests. Informal, qualitative comparisons with recognized baselines or benchmarks, and with general levels or trends in performance, must be clearly stated to be such and thus open to statistical reassessment.