discuss.gurock.com Open in urlscan Pro
2602:fd3f:3:ff02::2c  Public Scan

Form analysis
1 forms found in the DOM

POST /login

<form id="hidden-login-form" method="post" action="/login" style="display: none;">
  <input name="username" type="text" id="signin_username">
  <input name="password" type="password" id="signin_password">
  <input name="redirect" type="hidden">
  <input type="submit" id="signin-button" value="Log In">
</form>

Text Content

 * TestRail
 * Blog
 * Community

 * My Account

  This website uses cookies to ensure you get the best experience on our
website.
Yes, I consent No, Thanks
Skip to main content
Sign UpLog In
 * 
 * 


USING TESTRAIL ADD_RUN/UPDATE_RUN API CALLS FOR AUTOMATED TEST RUNS

TestRail Customizations


You have selected 0 posts.

select all

cancel selecting

Nov 2015
1 / 17
Nov 2015

Jun 2020

derek.wong
Nov '15


Hi there,

I’ve recently worked on integrating our Specflow based functional test framework
with TestRail so that it can create a test run and post results for the Gherkin
scenarios to that test run for reporting. It is working okay but the suite that
I create the test run from is not fully automated by Specflow. Currently when I
create the test run using add_run I set “include_all” to true. This means that
as I run the tests, those that are in that suite that are not automated remain
“Untested” (sensible in a way I guess) while the automated tests are set to pass
or fail as appropriate. Problem with this approach is that from my point of view
unless I move out all non-automated tests from this suite the results and
reports always a bit strange.

So I’m thinking of the approach where you add_run but set “include_all” to false
instead. What I’d like to confirm is the behaviour regarding “case_ids” - if you
start with an empty set “[]” in add_run is every subsequent update_run with
“case_ids” going to be the union of all case ids from previous invocations of
update_run? Or will I have to resend all previous case ids plus the new case id
that I want to add when I invoke update_run? The online documentation for
update_run wasn’t absolutely clear on this.

Hope I have explained myself clearly enough.

Thanks!

Regards,
Derek.





 * CREATED
   
   Nov '15

 * LAST REPLY
   
   Jun '20
 * 16
   
   REPLIES

 * 9.3k
   
   VIEWS

 * 10
   
   USERS

 * 1
   
   LIKE

 * 3
   
   LINKS

 * 6
   3
   


tgurock
Nov '15


Hi Derek,

Thanks for your posting! The case_ids field would be overridden in this case
(otherwise you couldn’t reset/empty this field via update_run if this would be
the union of all previous cases). We would recommend setting up the case
selection once when you create the test run and the typical approach would be as
follows for a custom selection:

 * Get a list of all case IDs for the test suite via get_cases (e.g. with a
   type_id filter or local filtering):
   http://docs.gurock.com/testrail-api2/reference-cases#get_cases 470
 * Then create a test run via add_run and include_all = false and pass the case
   IDs

This is also much more efficient than calling update_run for each test or
results and the recommended way to handle a custom selection via the API.

I hope this helps!

Cheers,
Tobias





28 days later
ckoz2015
Dec '15


I am also trying to add_run/update_run to create a run from one suite but also
add some cases from another suite. Is this possible?
Using this:
apstr = ‘add_run/’ + str(projectID)
result = client.send_post(apstr,{‘name’:testRunName,‘description’:‘Test suites
being run on QA’,‘suite_id’:suiteID,‘include_all’:True,‘case_ids’:[5567845]})

The one test case is from a different suite than that referenced in the suiteID.
It is a legitimate case suite. If I set include all = True, just the test cases
from the suite are included, if I set include all = False, no test cases are
added.

Likewise, if I don’t add any case numbers other than the test suite and try
update_run, no other test cases are added? Is this expected behavior?

Thanks,
Christi





tgurock
Dec '15


Hello Christi,

Thanks for your posting! A test run is always linked to a single test suite but
you can add a test plan to start multiple related test runs (also for different
suites):

http://docs.gurock.com/testrail-api2/reference-plans 201

If you set include_all to false, you would need to specify the entire case
selection via the case_ids attribute (expects an array of case IDs).

I hope this helps!

Cheers,
Tobias





3 months later
SRV
Feb '16


Hello-

I am trying to integrate Test Automation Framework with Test Rail. I am
exploring TestRail API V2 for integration. I am more particularly looking at
“add_run” call.
Using this api call I noticed that it creates a Test Run with Default Name
(“Master”) and includes all the test cases in the mentioned project ID, even
though “include_all” = false.
Here is the sample payload I am using:
{
“results”: [
{

        "name": "Automation Integration Test 1",
        "assignedto_id": 20,
        "include_all": false,
     "case_ids": [58086,45702]			

	}
]


}

Is this a know defect? Or could you please guide me on how to consume this API
call?

Thanks





tgurock
Feb '16


Hi!

You are using a slightly incorrect API request and you would need to specify the
name, assignedto_id etc. attributes directly on the root level of your JSON
payload. You grouped these attributes under an additional results attribute
instead and this convention is only used for the add_results and
add_results_for_cases API calls. The add_run call is much simpler and looks as
follows:

{
	"name": "This is a new test run",
	"assignedto_id": 5,
	"include_all": false,
	"case_ids": [1, 2, 3, 4, 7, 8]
}


I can also recommend using one of the API bindings instead of raw JSON/HTTP and
the bindings are currently available for Java, .NET, Python, PHP and Ruby:

http://docs.gurock.com/testrail-api2/start 118

I hope this helps!

Cheers,
Tobias





13 days later
lwyporek
Mar '16


Hi!

I am working on a similar mechanism for my automation to eliminate “Untested”
results within a test run.
I’ve come up with a solution that creates a test plan entry with
include_all=true and then incrementally updates case_ids of the test plan entry
(with POST to /update_plan_entry) each time it stores new test results. It works
fine provided I don’t use “Configurations”. Once I add config_ids when creating
a test plan entry, I no longer can eliminate “Untested” results because I’m
updating test plan entry but the configuration itself has it’s own set of test
cases, which by default is set to all. I can change it from UI but I can’t see
an API method to update this setting. Any thoughts on that?
Thanks!

Kind regards,
Lukasz





tgurock
Mar '16


Hello Lukasz,

Thanks for your feedback! We would usually recommend setting the case selection
before starting your automated tests (based on the set of tests you plan to
execute) and then going through the automated tests. Updating the case selection
for every result/tests is not very efficient and would involve extra API calls
and changing the case selection all the time. The ideal and fastest approach
would look as follows:

 * Create a test run or plan with include_all or a custom case selection (with
   the set of tests you want to run)
 * Use add_results or add_results_for_cases to add all test results (or maybe
   chunk-wise)

Cheers,
Tobias





lwyporek
2
Mar '16


Hi Tobias,

Thank you for your fast answer.
Updating case selection each time I store results is not an issue in my case
because I do it in bulks using add_results_for_cases (and not separately for
each case). So this is really few additional API calls per test run.
What I am missing in the API is the possibility to update case selection for a
configuration, so I don’t end up with a situation like below:


screenshot.png1635×192 21.4 KB



Kind regards,
Lukasz





tgurock
Mar '16


Hello Lukasz,

You can update the case selection for a test plan entry (update_plan_entry).
This will also set the case selection for the configurations, unless the
configuration has overridden the case selection. Updating an overridden case
selection is currently not directly possible via the API (only via the UI) but
it’s already planned to look into this. As a workaround, you could look into
adding plan entries with just a single configuration/run and using the
entry-level case selection. This way, you can update the case selection of the
plan entry and this updates the case selection of the configuration in the same
step (update_plan_entry). Would this work for you?

Cheers,
Tobias





lwyporek
1
Mar '16


Hi Tobias,

Thank you for the explanation. I’ve analysed the way I’m creating a test plan
entry and I found that the problem was that I was overriding the plan entry case
selection by including include_all=false in the actual run, like that:

{
    "suite_id": 33,
    "name": "test_50",
    "config_ids": [ 23, 26 ],
    "assignedto_id": 84,
    "include_all": false,
    "case_ids": [1],
    "description": "test_50 description",
    "runs": [
        {
            "assignedto_id": 84,
            "config_ids": [ 23, 26 ],
            "include_all": false
        }
    ]
}


Now, I’m setting include_all=false for the test plan entry only:

{
    "suite_id": 33,
    "name": "test_50",
    "config_ids": [ 23, 26 ],
    "assignedto_id": 84,
    "include_all": false,
    "case_ids": [1],
    "description": "test_50 description",
    "runs": [
        {
            "assignedto_id": 84,
            "config_ids": [ 23, 26 ]
        }
    ]
}


… and that works perfectly for me, because the configuration/run inherits the
case selection from the test plan entry:



Screen Shot 2016-03-09 at 09.10.26.png1627×178 22.4 KB



Thank you for your help!

Kind regards,
Lukasz





tgurock
Mar '16


Hello Lukasz,

That’s great to hear This is also what I meant but I thought you might be using
multiple configurations in one test plan entry and the screenshot you posted was
just an example. But it appears that you are already using just a single
configuration per test plan entry only so this approach works well.

Cheers,
Tobias





4 years later
Sachin
Sep '19


Is it possible to add a Test Run with all the test case in a group ?





20 days later
montu
Oct '19


Hi @tgurock

It has been 4 year since this feature is requested. Is this feature is
implemented or will implement in the near future? We want to add test case
filter by type_id during add_run process.
Currently, we can do by making include_all = false and typing case ids
[1,2,3…200…].
We have more than 200+ test cases with type_id= 3 so it’s not possible to add
run by typing case_ids = 1,2,3…200 every time.

Is there any way that it could be as simple as this:
" addRun = client.send_post(‘add_run/1’,
{‘suite_id’ : 2, ‘include_all’ :False, ‘type_id’ : 3}) "

Thanks,





jacob.scottGurock Team
Oct '19


Hey there,

Thanks for the feedback all. The ability to filter test cases with the add_run
API call is still not supported, but is still something we’re planning to
explore for a future release. I can’t give any specific timeframe for when this
would be available or if it will be implemented in the fashion you suggested
@montu, but for now I’ve added your feedback to the internal request to help the
development team prioritize new features and updates for upcoming releases.

@Sachin, while it wouldn’t be possible to set it up with a simple filter within
the add_run call, you can do a get_cases call with filter options by section,
type, or other supported filters and then feed the returned case IDs into the
test case selection of the add_run call.

Thanks again for the feedback, and if there is anything else we can assist with
feel free to reach out to the support team directly at contact@gurock.com.

1



2 months later
ohonc
2
Dec '19


Hello @jacob.scott i have not found, but may be u have some solution, i want to
create add_plan, but with specifying some of section id from this list of
sections GET index.php?/api/v2/get_sections/:project_id&suite_id=:suite_id, that
have "depth": 0. I mean have i some chance to add only some group of tests and
not using "case_ids": [1, 2, 3], ?





6 months later
vijaykumarbhalli
Jun '20


hi @tgurock i am getting null pointer exception for add_result_for_case for
below code

public void addResultForTestCases(String testCaseId, int status) throws
MalformedURLException, IOException, APIException
{
/*
* DateTimeFormatter dtf = DateTimeFormatter.ofPattern(“yyyy/MM/dd HH:mm:ss”);
* LocalDateTime now = LocalDateTime.now();
*
* APIClient client = new APIClient(“https://lunazone.testrail.io/”);
* client.setUser(“dplusin_awsdev@discovery.com”);
* client.setPassword(“DPlusdev01”);
*
* data = new HashMap(); data.put(“include_all”,true);
* data.put(“name”,"Android Automation Run "+dtf.format(now));
*
* JSONObject c = (JSONObject)client.sendPost(“add_run/”+PROJECT_ID,data); Long
* suiteID = (Long)c.get(“id”);
*
* System.out.println("Suite ID is "+suiteID);
*/
data = new HashMap();

	Long i = RunCukesTest.runID;
	
	
	
	if(status == 1) 
	{
		data.put("status_id", 1);
		data.put("comment", "Test Executed - Status updated automatically from Selenium test automation.");
	}
	else if(status == 5)
	{
		String imagename = "@C"+tagName+"-Error";
		data.put("status_id", 5);
		File src = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
		File screenShotLocation = new File(System.getProperty("user.dir")+"/target/"+imagename+".png");
		
		FileUtils.copyFile(src, screenShotLocation);
		
		client.sendPost("add_attachment_to_result/5", screenShotLocation);
		
		data.put("comment", "Test Executed - Test Case failed");
	}
	
		System.out.println("Data val is "+data);
		client.sendPost("add_result_for_case/:"+i+"/:"+testCaseId ,data);
}


please help me on same








Reply



SUGGESTED TOPICS

Topic Replies Views Activity Error on push to next gen project in JIRA
TestRail Customizations
0 57 Feb '21 [UI scripts] New dropdowns don’t fire ‘change’ event
TestRail Customizations
2 53 Jul '21 TestRail API - groups and users
TestRail Customizations
2 64 Mar '21 Copy a test case
TestRail Customizations
1 37 Apr '21 3rd party tools for visualization and analysis of Test Rail data
TestRail Customizations
0 43 Sep '21


WANT TO READ MORE? BROWSE OTHER TOPICS IN TESTRAIL CUSTOMIZATIONS OR VIEW LATEST
TOPICS.


[EN.SHARE.TOPIC]



Share



 * Contact
 * Security
 * Privacy Policy
 * Terms
 * © 2003-2021 Gurock


Invalid date Invalid date