CollectingTestListener

public class CollectingTestListener
extends Object implements ITestInvocationListener, ILogSaverListener

java.lang.Object
↳	com.android.tradefed.result.CollectingTestListener

Known direct subclasses EmailResultReporter, JsonHttpTestResultReporter, MetricsXMLResultReporter, ModuleListener, ResultAggregator, ShardListener, SuiteResultReporter, XmlResultReporter EmailResultReporter A simple result reporter base class that sends emails for test results.  JsonHttpTestResultReporter A result reporter that encode test metrics results and branch, device info into JSON and POST into an HTTP service endpoint   MetricsXMLResultReporter MetricsXMLResultReporter writes test metrics and run metrics to an XML file in a folder specified by metrics-folder parameter at the invocationEnded phase of the test.  ModuleListener Listener attached to each IRemoteTest of each module in order to collect the list of results.  ResultAggregator Special forwarder that aggregates the results when needed, based on the retry strategy that was taken.  ShardListener A ITestInvocationListener that collects results from a invocation shard (aka an invocation split to run on multiple resources in parallel), and forwards them to another listener.  SuiteResultReporter Collect test results for an entire suite invocation and output the final results.  XmlResultReporter Writes JUnit results to an XML files in a format consistent with Ant's XMLJUnitResultFormatter.

Known indirect subclasses DeviceUnavailEmailResultReporter, FailureEmailResultReporter, FormattedGeneratorReporter, InvocationFailureEmailResultReporter, TestFailureEmailResultReporter, XmlFormattedGeneratorReporter DeviceUnavailEmailResultReporter An EmailResultReporter that will send email when invocation fails due to a device not available exception.  FailureEmailResultReporter An EmailResultReporter that sends notifications if either a test failure or invocation failure occurred.  FormattedGeneratorReporter Reporter that allows to generate reports in a particular format.  InvocationFailureEmailResultReporter An EmailResultReporter that can also restrict notifications to just invocation failures.  TestFailureEmailResultReporter An EmailResultReporter that can also restrict notifications to just test failures.  XmlFormattedGeneratorReporter Implementation of the FormattedGeneratorReporter which format the suite results in an xml format.

---

A ITestInvocationListener that will collect all test results.

Although the data structures used in this object are thread-safe, the ITestInvocationListener callbacks must be called in the correct order.

Summary

Public constructors
CollectingTestListener()

Public methods
IBuildInfo	getBuildInfo() This method is deprecated. rely on the IBuildInfo from getInvocationContext().
TestRunResult	getCurrentRunResults() Gets the results for the current test run.
int	getExpectedTests() Returns the number of expected tests count.
IInvocationContext	getInvocationContext() Return the invocation context that was reported via invocationStarted(com.android.tradefed.invoker.IInvocationContext)
	getMergedTestRunResults() Return the merged collection of results for all runs across different attempts.
IInvocationContext	getModuleContextForRunResult(String testRunName) Returns the IInvocationContext of the module associated with the results.
	getNonAssociatedLogFiles() Returns a copy of the map containing all the logged file not associated with a test run.
int	getNumAllFailedTestRuns() Returns the total number of test runs in a failure state
int	getNumAllFailedTests() Returns the total number of tests in a failure state (only failed, assumption failures do not count toward it).
int	getNumTestsInState(TestResult.TestStatus status) Returns the number of tests in given state for this run.
int	getNumTotalTests() Returns the total number of complete tests for all runs.
IBuildInfo	getPrimaryBuildInfo() Return the primary build info that was reported via invocationStarted(com.android.tradefed.invoker.IInvocationContext).
	getRunResults() This method is deprecated. Use getMergedTestRunResults()
TestRunResult	getTestRunAtAttempt(String testRunName, int attempt) Return the TestRunResult for a single attempt.
int	getTestRunAttemptCount(String testRunName) Returns the number of attempts for a given test run name.
	getTestRunAttempts(String testRunName) Gets all the attempts for a TestRunResult of a given test run.
	getTestRunForAttempts(int attempt) Gets all the results for a given attempt.
	getTestRunNames() Return all the names for all the test runs.
boolean	hasFailedTests() Returns if the invocation had any failed or assumption failed tests.
boolean	hasTestRunResultsForName(String testRunName) Returns whether a given test run name has any results.
void	invocationEnded(long elapsedTime)
void	invocationFailed(Throwable cause)
void	invocationStarted(IInvocationContext context)
void	logAssociation(String dataName, LogFile logFile)
void	setBuildInfo(IBuildInfo buildInfo) This method is deprecated. Not necessary for testing anymore.
void	setMergeStrategy(MergeStrategy strategy) Sets the MergeStrategy to use when merging results.
void	testAssumptionFailure(TestDescription test, String trace)
void	testAssumptionFailure(TestDescription test, FailureDescription failure)
void	testEnded(TestDescription test, long endTime, testMetrics)
void	testEnded(TestDescription test, testMetrics)
void	testFailed(TestDescription test, FailureDescription failure)
void	testFailed(TestDescription test, String trace)
void	testIgnored(TestDescription test)
void	testModuleEnded()
void	testModuleStarted(IInvocationContext moduleContext)
void	testRunEnded(long elapsedTime, runMetrics)
void	testRunFailed(FailureDescription failure)
void	testRunFailed(String errorMessage)
void	testRunStarted(String name, int numTests, int attemptNumber)
void	testRunStarted(String name, int numTests)
void	testRunStarted(String name, int numTests, int attemptNumber, long startTime)
void	testRunStopped(long elapsedTime)
void	testStarted(TestDescription test, long startTime)
void	testStarted(TestDescription test)

Protected methods
final void	clearResultsForName(String testRunName) Allows to clear the results for a given run name.
void	setIsAggregrateMetrics(boolean aggregate) Toggle the 'aggregate metrics' option

Public constructors

CollectingTestListener

public CollectingTestListener ()

Public methods

getBuildInfo

public IBuildInfo getBuildInfo ()

This method is deprecated.
rely on the IBuildInfo from getInvocationContext().

Returns the build info.

Returns
IBuildInfo

getCurrentRunResults

public TestRunResult getCurrentRunResults ()

Gets the results for the current test run.

Note the results may not be complete. It is recommended to test the value of TestRunResult#isRunComplete() and/or (@link TestRunResult#isRunFailure()} as appropriate before processing the results.

Returns
TestRunResult	the TestRunResult representing data collected during last test run

getExpectedTests

public int getExpectedTests ()

Returns the number of expected tests count. Could differ from getNumTotalTests() if some tests did not run.

Returns
int

getInvocationContext

public IInvocationContext getInvocationContext ()

Return the invocation context that was reported via invocationStarted(com.android.tradefed.invoker.IInvocationContext)

Returns
IInvocationContext

getMergedTestRunResults

public  getMergedTestRunResults ()

Return the merged collection of results for all runs across different attempts.

If there are multiple results, each test run is merged, with the latest test result overwriting test results of previous runs. Test runs are ordered by attempt number.

Metrics for the same attempt will be merged based on the preference set by aggregate-metrics. The final metrics will be the metrics of the last attempt.

Returns

getModuleContextForRunResult

public IInvocationContext getModuleContextForRunResult (String testRunName)

Returns the IInvocationContext of the module associated with the results.

Parameters
testRunName	String: The name given by {testRunStarted(String, int).

Returns
IInvocationContext	The IInvocationContext of the module for a given test run name null if there are no results for that name.

getNonAssociatedLogFiles

public  getNonAssociatedLogFiles ()

Returns a copy of the map containing all the logged file not associated with a test run.

Returns

getNumAllFailedTestRuns

public int getNumAllFailedTestRuns ()

Returns the total number of test runs in a failure state

Returns
int

getNumAllFailedTests

public int getNumAllFailedTests ()

Returns the total number of tests in a failure state (only failed, assumption failures do not count toward it).

Returns
int

getNumTestsInState

public int getNumTestsInState (TestResult.TestStatus status)

Returns the number of tests in given state for this run.

Parameters
status	TestResult.TestStatus

Returns
int

getNumTotalTests

public int getNumTotalTests ()

Returns the total number of complete tests for all runs.

Returns
int

getPrimaryBuildInfo

public IBuildInfo getPrimaryBuildInfo ()

Return the primary build info that was reported via invocationStarted(com.android.tradefed.invoker.IInvocationContext). Primary build is the build returned by the first build provider of the running configuration. Returns null if there is no context (no build to test case).

Returns
IBuildInfo

getRunResults

public  getRunResults ()

This method is deprecated.
Use getMergedTestRunResults()

Returns the results for all test runs.

Returns

getTestRunAtAttempt

public TestRunResult getTestRunAtAttempt (String testRunName, 
                int attempt)

Return the TestRunResult for a single attempt.

Parameters
testRunName	String: The name given by {testRunStarted(String, int).
attempt	int: The attempt id.

Returns
TestRunResult	The TestRunResult for the given name and attempt id or null if it does not exist.

getTestRunAttemptCount

public int getTestRunAttemptCount (String testRunName)

Returns the number of attempts for a given test run name.

Parameters
testRunName	String: The name given by {testRunStarted(String, int).

Returns
int

getTestRunAttempts

public  getTestRunAttempts (String testRunName)

Gets all the attempts for a TestRunResult of a given test run.

Parameters
testRunName	String: The name given by {testRunStarted(String, int).

Returns
	All TestRunResult for a given test run, ordered by attempts.

getTestRunForAttempts

public  getTestRunForAttempts (int attempt)

Gets all the results for a given attempt.

Parameters
attempt	int: The attempt we want results for.

Returns
	All TestRunResult for a given attempt.

getTestRunNames

public  getTestRunNames ()

Return all the names for all the test runs.

These test runs may have run multiple times with different attempts.

Returns

hasFailedTests

public boolean hasFailedTests ()

Returns if the invocation had any failed or assumption failed tests.

Returns
boolean

hasTestRunResultsForName

public boolean hasTestRunResultsForName (String testRunName)

Returns whether a given test run name has any results.

Parameters
testRunName	String: The name given by {testRunStarted(String, int).

Returns
boolean

invocationEnded

public void invocationEnded (long elapsedTime)

Parameters
elapsedTime	long

invocationFailed

public void invocationFailed (Throwable cause)

Parameters
cause	Throwable

invocationStarted

public void invocationStarted (IInvocationContext context)

Parameters
context	IInvocationContext

logAssociation

public void logAssociation (String dataName, 
                LogFile logFile)

Parameters
dataName	String
logFile	LogFile

setBuildInfo

public void setBuildInfo (IBuildInfo buildInfo)

This method is deprecated.
Not necessary for testing anymore.

Set the build info. Should only be used for testing.

Parameters
buildInfo	IBuildInfo

setMergeStrategy

public void setMergeStrategy (MergeStrategy strategy)

Sets the MergeStrategy to use when merging results.

Parameters
strategy	MergeStrategy

testAssumptionFailure

public void testAssumptionFailure (TestDescription test, 
                String trace)

Parameters
test	TestDescription
trace	String

testAssumptionFailure

public void testAssumptionFailure (TestDescription test, 
                FailureDescription failure)

Parameters
test	TestDescription
failure	FailureDescription

testEnded

public void testEnded (TestDescription test, 
                long endTime, 
                 testMetrics)

Parameters
test	TestDescription
endTime	long
testMetrics

testEnded

public void testEnded (TestDescription test, 
                 testMetrics)

Parameters
test	TestDescription
testMetrics

testFailed

public void testFailed (TestDescription test, 
                FailureDescription failure)

Parameters
test	TestDescription
failure	FailureDescription

testFailed

public void testFailed (TestDescription test, 
                String trace)

Parameters
test	TestDescription
trace	String

testIgnored

public void testIgnored (TestDescription test)

Parameters
test	TestDescription

testModuleEnded

public void testModuleEnded ()

testModuleStarted

public void testModuleStarted (IInvocationContext moduleContext)

Parameters
moduleContext	IInvocationContext

testRunEnded

public void testRunEnded (long elapsedTime, 
                 runMetrics)

Parameters
elapsedTime	long
runMetrics

testRunFailed

public void testRunFailed (FailureDescription failure)

Parameters
failure	FailureDescription

testRunFailed

public void testRunFailed (String errorMessage)

Parameters
errorMessage	String

testRunStarted

public void testRunStarted (String name, 
                int numTests, 
                int attemptNumber)

Parameters
name	String
numTests	int
attemptNumber	int

testRunStarted

public void testRunStarted (String name, 
                int numTests)

Parameters
name	String
numTests	int

testRunStarted

public void testRunStarted (String name, 
                int numTests, 
                int attemptNumber, 
                long startTime)

Parameters
name	String
numTests	int
attemptNumber	int
startTime	long

testRunStopped

public void testRunStopped (long elapsedTime)

Parameters
elapsedTime	long

testStarted

public void testStarted (TestDescription test, 
                long startTime)

Parameters
test	TestDescription
startTime	long

testStarted

public void testStarted (TestDescription test)

Parameters
test	TestDescription

Protected methods

clearResultsForName

protected final void clearResultsForName (String testRunName)

Allows to clear the results for a given run name. Should only be used in some cases like the aggregator of results.

Parameters
testRunName	String

setIsAggregrateMetrics

protected void setIsAggregrateMetrics (boolean aggregate)

Toggle the 'aggregate metrics' option

Parameters
aggregate	boolean