Evaluate
########

* A library for easily evaluating machine learning models and datasets.


Tutorials
=========

Installation
------------

pip安装::

	pip install evaluate

	# check if 🤗Evaluate has been properly installed
	python -c "import evaluate; print(evaluate.load('exact_match').compute(references=['hello'], predictions=['hello']))"

源码安装::

	git clone https://github.com/huggingface/evaluate.git
	cd evaluate
	pip install -e .




A quick tour
------------

Load::

	import evaluate
	accuracy = evaluate.load("accuracy")

	# List available modules
	>>> evaluate.list_evaluation_modules(
	...   module_type="comparison",
	...   include_community=False,
	...   with_details=True)
	[{'name': 'mcnemar', 'type': 'comparison', 'community': False, 'likes': 1},
 	{'name': 'exact_match', 'type': 'comparison', 'community': False, 'likes': 0}]

Module attribute::

	>>> accuracy.description
	Accuracy is the proportion of correct predictions among the total number of cases processed. It can be computed with:
	Accuracy = (TP + TN) / (TP + TN + FP + FN)
	 Where:
	TP: True positive
	TN: True negative
	FP: False positive
	FN: False negative

	>>> accuracy.citation
	title={Scikit-learn: Machine Learning in {P}ython},
	  author={Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V.
	         and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P.
	         and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and
	         Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E.},
	  journal={Journal of Machine Learning Research},
	  volume={12},
	  pages={2825--2830},
	  year={2011}
	}

	>>> accuracy.features
	{
	    'predictions': Value(dtype='int32', id=None),
	    'references': Value(dtype='int32', id=None)
	}

How to compute::

	>>> accuracy.compute(references=[0,1,0,1], predictions=[1,0,0,1])
	{'accuracy': 0.5}



How-To Guide
============