fb48ecb57f
* V11: add test flakyness reporting (#13386) * update json-models-builders * Add reporting to all before each loops * Add commitId to env for pipelines * Test reports correctly * Bumb testhelper version * Update playwright-testhelpers version * Bumb playwright test-helper version * Bumb playwright-testhelpers version * Add AgentOs to yaml env * Bumb playwright-testhelper version * Try different AgentOS * Revert "Try different AgentOS" This reverts commit fc90620aef7af01ceffcdfbea7c9ef4dd02007f3. * Revert logo test * Add missing semicolons * Update package-lock.json * Add reports to Blocklist tests Co-authored-by: Zeegaan <nge@umbraco.dk> * V10: AllowedUploadFiles appsetting not working (#13408) * Add new Settings * Use new settings instead of old ones * Implement AllowedUploadedFiles value to be copied to AllowedUplayedFileExtensions * Obsolete old settings * Rename DisallowedUploadFileExtensions * Implement same fix for DisallowedUploadFiles * Use new settings for backoffice server variables * Update the correct setting Co-authored-by: Zeegaan <nge@umbraco.dk> * V11: Add compatibility suppressions (#13425) * Add suprressions * Revert "Add suprressions" This reverts commit 1c924ee967375870b6d86be9e9480ced0e22f8c7. * Update CompatibilitySuppressions.xml Co-authored-by: Ronald Barendse <ronald@barend.se> * Bump socket.io-parser from 4.0.4 to 4.0.5 in /src/Umbraco.Web.UI.Client Bumps [socket.io-parser](https://github.com/socketio/socket.io-parser) from 4.0.4 to 4.0.5. - [Release notes](https://github.com/socketio/socket.io-parser/releases) - [Changelog](https://github.com/socketio/socket.io-parser/blob/main/CHANGELOG.md) - [Commits](https://github.com/socketio/socket.io-parser/compare/4.0.4...4.0.5) --- updated-dependencies: - dependency-name: socket.io-parser dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * update block grid panel name (#13325) Co-authored-by: Jacob Overgaard <752371+iOvergaard@users.noreply.github.com> * Bump async from 2.6.3 to 2.6.4 in /src/Umbraco.Web.UI.Client Bumps [async](https://github.com/caolan/async) from 2.6.3 to 2.6.4. - [Release notes](https://github.com/caolan/async/releases) - [Changelog](https://github.com/caolan/async/blob/v2.6.4/CHANGELOG.md) - [Commits](https://github.com/caolan/async/compare/v2.6.3...v2.6.4) --- updated-dependencies: - dependency-name: async dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Bump eventsource from 1.1.0 to 1.1.2 in /src/Umbraco.Web.UI.Client Bumps [eventsource](https://github.com/EventSource/eventsource) from 1.1.0 to 1.1.2. - [Release notes](https://github.com/EventSource/eventsource/releases) - [Changelog](https://github.com/EventSource/eventsource/blob/master/HISTORY.md) - [Commits](https://github.com/EventSource/eventsource/compare/v1.1.0...v1.1.2) --- updated-dependencies: - dependency-name: eventsource dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Bump minimist from 1.2.5 to 1.2.7 in /src/Umbraco.Web.UI.Client Bumps [minimist](https://github.com/minimistjs/minimist) from 1.2.5 to 1.2.7. - [Release notes](https://github.com/minimistjs/minimist/releases) - [Changelog](https://github.com/minimistjs/minimist/blob/main/CHANGELOG.md) - [Commits](https://github.com/minimistjs/minimist/compare/v1.2.5...v1.2.7) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Management API: SearchController improvements (#13420) * remove typo of double PagedViewModel in response * add providerProperties with custom formatter * use a more descriptive variable name * use a more descriptive variable name for property types * generate new schema Co-authored-by: Zeegaan <nge@umbraco.dk> * Bump minimatch from 3.0.4 to 3.1.2 in /src/Umbraco.Web.UI.Client Bumps [minimatch](https://github.com/isaacs/minimatch) from 3.0.4 to 3.1.2. - [Release notes](https://github.com/isaacs/minimatch/releases) - [Commits](https://github.com/isaacs/minimatch/compare/v3.0.4...v3.1.2) --- updated-dependencies: - dependency-name: minimatch dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Zeegaan <nge@umbraco.dk> Co-authored-by: Mole <nikolajlauridsen@protonmail.ch> Co-authored-by: Ronald Barendse <ronald@barend.se> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Blake Watt <bkclerke@users.noreply.github.com> Co-authored-by: Jacob Overgaard <752371+iOvergaard@users.noreply.github.com>
Umbraco Acceptance Tests
You can watch a video following these instructions here and a longer UmbraCollab recording here. Make sure to use the latest recommended contribution branch rather than v10 that's mentioned in the video. Alternatively, follow along the instructions below.
Prerequisites
- NodeJS 16+
- A running installed Umbraco on url: https://localhost:44331 (Default development port)
- Install using a
SqlServer/LocalDbas the tests execute too fast forSqliteto handle.
- Install using a
Getting started
The tests are located in the project/folder as Umbraco.Tests.AcceptanceTests. Make sure you run npm ci & npx playwright install in that folder, or let your IDE do that.
The script will ask you to enter the username and password for a superadmin user of your Umbraco CMS.
Executing tests
There are two npm scripts that can be used to execute the test:
npm run test- Executes the tests headless.
npm run ui- Executes the tests in a browser handled by a playwright application.
In case of errors it is recommended to use await page.pause() so you can step through your test.
Executing single tests
If you wish to run a single test, which may be helpful when writing tests you can use the following command. As before, you need to run these tests in the 'tests/Umbraco.Tests.AcceptanceTest' folder.
npx playwright test <testname.ts>
For example to run the Login Test,
npx playwright test tests/DefaultConfig/Login/Login.spec.ts
Environment Configuration
The environment configuration is begin setup by the npm installation script.
This results in the creation of this file: .env.
This file is already added to .gitignore and can contain values that are different for each developer machine.
The file has the following content:
UMBRACO_USER_LOGIN=email for superadmin
UMBRACO_USER_PASSWORD=password for superadmin
URL=https://localhost:44331
You can change this if you like or run the config script to reset the values, type "npm run config" in your terminal.
Documentation
For further documentation on Playwright, see the Playwright documentation.