discuss.gurock.com
Open in
urlscan Pro
2602:fd3f:3:ff02::2c
Public Scan
URL:
https://discuss.gurock.com/t/using-testrail-add-run-update-run-api-calls-for-automated-test-runs/2537
Submission Tags: falconsandbox
Submission: On January 13 via api from US — Scanned from DE
Submission Tags: falconsandbox
Submission: On January 13 via api from US — Scanned from DE
Form analysis
1 forms found in the DOMPOST /login
<form id="hidden-login-form" method="post" action="/login" style="display: none;">
<input name="username" type="text" id="signin_username">
<input name="password" type="password" id="signin_password">
<input name="redirect" type="hidden">
<input type="submit" id="signin-button" value="Log In">
</form>
Text Content
* TestRail * Blog * Community * My Account This website uses cookies to ensure you get the best experience on our website. Yes, I consent No, Thanks Skip to main content Sign UpLog In * * USING TESTRAIL ADD_RUN/UPDATE_RUN API CALLS FOR AUTOMATED TEST RUNS TestRail Customizations You have selected 0 posts. select all cancel selecting Nov 2015 1 / 17 Nov 2015 Jun 2020 derek.wong Nov '15 Hi there, I’ve recently worked on integrating our Specflow based functional test framework with TestRail so that it can create a test run and post results for the Gherkin scenarios to that test run for reporting. It is working okay but the suite that I create the test run from is not fully automated by Specflow. Currently when I create the test run using add_run I set “include_all” to true. This means that as I run the tests, those that are in that suite that are not automated remain “Untested” (sensible in a way I guess) while the automated tests are set to pass or fail as appropriate. Problem with this approach is that from my point of view unless I move out all non-automated tests from this suite the results and reports always a bit strange. So I’m thinking of the approach where you add_run but set “include_all” to false instead. What I’d like to confirm is the behaviour regarding “case_ids” - if you start with an empty set “[]” in add_run is every subsequent update_run with “case_ids” going to be the union of all case ids from previous invocations of update_run? Or will I have to resend all previous case ids plus the new case id that I want to add when I invoke update_run? The online documentation for update_run wasn’t absolutely clear on this. Hope I have explained myself clearly enough. Thanks! Regards, Derek. * CREATED Nov '15 * LAST REPLY Jun '20 * 16 REPLIES * 9.3k VIEWS * 10 USERS * 1 LIKE * 3 LINKS * 6 3 tgurock Nov '15 Hi Derek, Thanks for your posting! The case_ids field would be overridden in this case (otherwise you couldn’t reset/empty this field via update_run if this would be the union of all previous cases). We would recommend setting up the case selection once when you create the test run and the typical approach would be as follows for a custom selection: * Get a list of all case IDs for the test suite via get_cases (e.g. with a type_id filter or local filtering): http://docs.gurock.com/testrail-api2/reference-cases#get_cases 470 * Then create a test run via add_run and include_all = false and pass the case IDs This is also much more efficient than calling update_run for each test or results and the recommended way to handle a custom selection via the API. I hope this helps! Cheers, Tobias 28 days later ckoz2015 Dec '15 I am also trying to add_run/update_run to create a run from one suite but also add some cases from another suite. Is this possible? Using this: apstr = ‘add_run/’ + str(projectID) result = client.send_post(apstr,{‘name’:testRunName,‘description’:‘Test suites being run on QA’,‘suite_id’:suiteID,‘include_all’:True,‘case_ids’:[5567845]}) The one test case is from a different suite than that referenced in the suiteID. It is a legitimate case suite. If I set include all = True, just the test cases from the suite are included, if I set include all = False, no test cases are added. Likewise, if I don’t add any case numbers other than the test suite and try update_run, no other test cases are added? Is this expected behavior? Thanks, Christi tgurock Dec '15 Hello Christi, Thanks for your posting! A test run is always linked to a single test suite but you can add a test plan to start multiple related test runs (also for different suites): http://docs.gurock.com/testrail-api2/reference-plans 201 If you set include_all to false, you would need to specify the entire case selection via the case_ids attribute (expects an array of case IDs). I hope this helps! Cheers, Tobias 3 months later SRV Feb '16 Hello- I am trying to integrate Test Automation Framework with Test Rail. I am exploring TestRail API V2 for integration. I am more particularly looking at “add_run” call. Using this api call I noticed that it creates a Test Run with Default Name (“Master”) and includes all the test cases in the mentioned project ID, even though “include_all” = false. Here is the sample payload I am using: { “results”: [ { "name": "Automation Integration Test 1", "assignedto_id": 20, "include_all": false, "case_ids": [58086,45702] } ] } Is this a know defect? Or could you please guide me on how to consume this API call? Thanks tgurock Feb '16 Hi! You are using a slightly incorrect API request and you would need to specify the name, assignedto_id etc. attributes directly on the root level of your JSON payload. You grouped these attributes under an additional results attribute instead and this convention is only used for the add_results and add_results_for_cases API calls. The add_run call is much simpler and looks as follows: { "name": "This is a new test run", "assignedto_id": 5, "include_all": false, "case_ids": [1, 2, 3, 4, 7, 8] } I can also recommend using one of the API bindings instead of raw JSON/HTTP and the bindings are currently available for Java, .NET, Python, PHP and Ruby: http://docs.gurock.com/testrail-api2/start 118 I hope this helps! Cheers, Tobias 13 days later lwyporek Mar '16 Hi! I am working on a similar mechanism for my automation to eliminate “Untested” results within a test run. I’ve come up with a solution that creates a test plan entry with include_all=true and then incrementally updates case_ids of the test plan entry (with POST to /update_plan_entry) each time it stores new test results. It works fine provided I don’t use “Configurations”. Once I add config_ids when creating a test plan entry, I no longer can eliminate “Untested” results because I’m updating test plan entry but the configuration itself has it’s own set of test cases, which by default is set to all. I can change it from UI but I can’t see an API method to update this setting. Any thoughts on that? Thanks! Kind regards, Lukasz tgurock Mar '16 Hello Lukasz, Thanks for your feedback! We would usually recommend setting the case selection before starting your automated tests (based on the set of tests you plan to execute) and then going through the automated tests. Updating the case selection for every result/tests is not very efficient and would involve extra API calls and changing the case selection all the time. The ideal and fastest approach would look as follows: * Create a test run or plan with include_all or a custom case selection (with the set of tests you want to run) * Use add_results or add_results_for_cases to add all test results (or maybe chunk-wise) Cheers, Tobias lwyporek 2 Mar '16 Hi Tobias, Thank you for your fast answer. Updating case selection each time I store results is not an issue in my case because I do it in bulks using add_results_for_cases (and not separately for each case). So this is really few additional API calls per test run. What I am missing in the API is the possibility to update case selection for a configuration, so I don’t end up with a situation like below: screenshot.png1635×192 21.4 KB Kind regards, Lukasz tgurock Mar '16 Hello Lukasz, You can update the case selection for a test plan entry (update_plan_entry). This will also set the case selection for the configurations, unless the configuration has overridden the case selection. Updating an overridden case selection is currently not directly possible via the API (only via the UI) but it’s already planned to look into this. As a workaround, you could look into adding plan entries with just a single configuration/run and using the entry-level case selection. This way, you can update the case selection of the plan entry and this updates the case selection of the configuration in the same step (update_plan_entry). Would this work for you? Cheers, Tobias lwyporek 1 Mar '16 Hi Tobias, Thank you for the explanation. I’ve analysed the way I’m creating a test plan entry and I found that the problem was that I was overriding the plan entry case selection by including include_all=false in the actual run, like that: { "suite_id": 33, "name": "test_50", "config_ids": [ 23, 26 ], "assignedto_id": 84, "include_all": false, "case_ids": [1], "description": "test_50 description", "runs": [ { "assignedto_id": 84, "config_ids": [ 23, 26 ], "include_all": false } ] } Now, I’m setting include_all=false for the test plan entry only: { "suite_id": 33, "name": "test_50", "config_ids": [ 23, 26 ], "assignedto_id": 84, "include_all": false, "case_ids": [1], "description": "test_50 description", "runs": [ { "assignedto_id": 84, "config_ids": [ 23, 26 ] } ] } … and that works perfectly for me, because the configuration/run inherits the case selection from the test plan entry: Screen Shot 2016-03-09 at 09.10.26.png1627×178 22.4 KB Thank you for your help! Kind regards, Lukasz tgurock Mar '16 Hello Lukasz, That’s great to hear This is also what I meant but I thought you might be using multiple configurations in one test plan entry and the screenshot you posted was just an example. But it appears that you are already using just a single configuration per test plan entry only so this approach works well. Cheers, Tobias 4 years later Sachin Sep '19 Is it possible to add a Test Run with all the test case in a group ? 20 days later montu Oct '19 Hi @tgurock It has been 4 year since this feature is requested. Is this feature is implemented or will implement in the near future? We want to add test case filter by type_id during add_run process. Currently, we can do by making include_all = false and typing case ids [1,2,3…200…]. We have more than 200+ test cases with type_id= 3 so it’s not possible to add run by typing case_ids = 1,2,3…200 every time. Is there any way that it could be as simple as this: " addRun = client.send_post(‘add_run/1’, {‘suite_id’ : 2, ‘include_all’ :False, ‘type_id’ : 3}) " Thanks, jacob.scottGurock Team Oct '19 Hey there, Thanks for the feedback all. The ability to filter test cases with the add_run API call is still not supported, but is still something we’re planning to explore for a future release. I can’t give any specific timeframe for when this would be available or if it will be implemented in the fashion you suggested @montu, but for now I’ve added your feedback to the internal request to help the development team prioritize new features and updates for upcoming releases. @Sachin, while it wouldn’t be possible to set it up with a simple filter within the add_run call, you can do a get_cases call with filter options by section, type, or other supported filters and then feed the returned case IDs into the test case selection of the add_run call. Thanks again for the feedback, and if there is anything else we can assist with feel free to reach out to the support team directly at contact@gurock.com. 1 2 months later ohonc 2 Dec '19 Hello @jacob.scott i have not found, but may be u have some solution, i want to create add_plan, but with specifying some of section id from this list of sections GET index.php?/api/v2/get_sections/:project_id&suite_id=:suite_id, that have "depth": 0. I mean have i some chance to add only some group of tests and not using "case_ids": [1, 2, 3], ? 6 months later vijaykumarbhalli Jun '20 hi @tgurock i am getting null pointer exception for add_result_for_case for below code public void addResultForTestCases(String testCaseId, int status) throws MalformedURLException, IOException, APIException { /* * DateTimeFormatter dtf = DateTimeFormatter.ofPattern(“yyyy/MM/dd HH:mm:ss”); * LocalDateTime now = LocalDateTime.now(); * * APIClient client = new APIClient(“https://lunazone.testrail.io/”); * client.setUser(“dplusin_awsdev@discovery.com”); * client.setPassword(“DPlusdev01”); * * data = new HashMap(); data.put(“include_all”,true); * data.put(“name”,"Android Automation Run "+dtf.format(now)); * * JSONObject c = (JSONObject)client.sendPost(“add_run/”+PROJECT_ID,data); Long * suiteID = (Long)c.get(“id”); * * System.out.println("Suite ID is "+suiteID); */ data = new HashMap(); Long i = RunCukesTest.runID; if(status == 1) { data.put("status_id", 1); data.put("comment", "Test Executed - Status updated automatically from Selenium test automation."); } else if(status == 5) { String imagename = "@C"+tagName+"-Error"; data.put("status_id", 5); File src = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE); File screenShotLocation = new File(System.getProperty("user.dir")+"/target/"+imagename+".png"); FileUtils.copyFile(src, screenShotLocation); client.sendPost("add_attachment_to_result/5", screenShotLocation); data.put("comment", "Test Executed - Test Case failed"); } System.out.println("Data val is "+data); client.sendPost("add_result_for_case/:"+i+"/:"+testCaseId ,data); } please help me on same Reply SUGGESTED TOPICS Topic Replies Views Activity Error on push to next gen project in JIRA TestRail Customizations 0 57 Feb '21 [UI scripts] New dropdowns don’t fire ‘change’ event TestRail Customizations 2 53 Jul '21 TestRail API - groups and users TestRail Customizations 2 64 Mar '21 Copy a test case TestRail Customizations 1 37 Apr '21 3rd party tools for visualization and analysis of Test Rail data TestRail Customizations 0 43 Sep '21 WANT TO READ MORE? BROWSE OTHER TOPICS IN TESTRAIL CUSTOMIZATIONS OR VIEW LATEST TOPICS. [EN.SHARE.TOPIC] Share * Contact * Security * Privacy Policy * Terms * © 2003-2021 Gurock Invalid date Invalid date