2020-06-17 16:39:28 +02:00
|
|
|
using System;
|
|
|
|
|
using System.Linq.Expressions;
|
|
|
|
|
using System.Net.Http;
|
2021-10-19 10:09:48 +01:00
|
|
|
using System.Reflection;
|
2020-09-02 18:10:29 +10:00
|
|
|
using Microsoft.AspNetCore.Authentication;
|
|
|
|
|
using Microsoft.AspNetCore.Builder;
|
|
|
|
|
using Microsoft.AspNetCore.Hosting;
|
2020-06-17 16:39:28 +02:00
|
|
|
using Microsoft.AspNetCore.Http;
|
|
|
|
|
using Microsoft.AspNetCore.Mvc.Testing;
|
|
|
|
|
using Microsoft.AspNetCore.Routing;
|
2020-09-02 18:10:29 +10:00
|
|
|
using Microsoft.AspNetCore.TestHost;
|
2022-02-11 14:24:45 +00:00
|
|
|
using Microsoft.Extensions.Configuration;
|
2020-06-17 16:39:28 +02:00
|
|
|
using Microsoft.Extensions.DependencyInjection;
|
2020-12-21 15:58:47 +11:00
|
|
|
using Microsoft.Extensions.Hosting;
|
2022-02-11 14:24:45 +00:00
|
|
|
using Moq;
|
2020-06-17 16:39:28 +02:00
|
|
|
using NUnit.Framework;
|
2021-02-18 11:06:02 +01:00
|
|
|
using Umbraco.Cms.Core.Cache;
|
|
|
|
|
using Umbraco.Cms.Core.Composing;
|
|
|
|
|
using Umbraco.Cms.Core.DependencyInjection;
|
2022-02-11 14:24:45 +00:00
|
|
|
using Umbraco.Cms.Core.Services;
|
2021-02-18 11:06:02 +01:00
|
|
|
using Umbraco.Cms.Core.Web;
|
v10 SQLite support + distributed locking abstractions (#11922)
* Created Persistence.SQLite project skeleton.
* SQLite database initialization
* Various changes and hacks to make things work.
* WIP integration tests
* Fix thread safety tests
* Fix tests that relied on tie breaker sorting.
Spent a fair amount of time looking for a less lazy fix but gave up.
* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join
* Fix test Can_Generate_Delete_SubQuery_Statement
Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].
SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."
Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.
* Fixes for paginated custom order by
* Fix tests broken by lack of unique indexes.
* Fix SqlServerTableByTableTest tests.
These tests didn't actually do anything as the tables already exist so schema creator just returned.
Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.
* Fix ValidateLoginSession - SelectTop must come later
* dry up database cleanup
* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.
* Prevent schema creation errors.
* SQLite ignore lock tests, WAL back on.
* Fix package schema tests
* Fix NPocoFetchTests - case sensitivity not under test
* Fix AdvancedMigrationTests (where possible)
Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.
If we do that can do some cleanup.
* Cleanup test database configuration
* Run integration tests against SQLite on build agent.
* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping
* YAML
* Skip Umbraco.Tests.Integration.SqlCe
* Drop SqlServerTableByTable tests.
Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).
Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors
* Might aswell run against macOS also.
* Copy azure pipelines task header layout
* Delete SQLCe projects
* Remove SQL CE specific code.
* Remove SQL CE NuSpec, template params, build script setup
* Delete umbraco-netcore-only.sln
* Add SkipTests solution configuration and use for codeql
* Remove reference to deleted nuspec file.
* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.
At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.
"ConnectionStrings": {
"umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
"umbracoDbDSN_ProviderName": "System.Data.SQLite"
},
Not currently possible via installer UI without provider name pre-set in
configuration.
* Switch to Microsoft.Data.Sqlite
Some gross hacks but will be good to find out if this works
with apple silicon.
* Enable selection of SQLite via installer UI (also quick install)
* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO
* Move SQL Server support to its own class library
* Add persistence dependencies to Umbraco.CMS metapackage
* Bugfix packages delete query
Created invalid query for SQLite.
* Try out cypress tests Linux + SQLite
* Prevent cypress test artifact upload failure on attempt 2+
* LocalDb bugfixes
* Drop redundant enum
* Move SqlClient constant
* Misc whitespace
* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).
* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*
* Display quick install defaults and per provider default database name.
* Misc remove old comment
* little re-arrange
* Remove almost all usages of IsSqlite extension.
* visual adjustments
* Custom Database Configuration is last step and should then say Install.
* use text instead of disabled inputs
* move legend, rename to Install
* Update SqlMainDomLock to work without distributed locks.
* Added IDistributedLockingMechanism interface and in memory impl.
* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.
* Added SqlServerDistributedLockingMechanism
* Move distributed locking interfaces and exceptions to Core + xmldocs.
* Fix tests, Misc cleanup, Add SQL distributed locking integration tests
* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)
* Nomplementation -> NoImplementation
* Fix misleading comment
* Integration tests use SqlServerDistributedLockingMechanism when possible
* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.
Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests
* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.
* Tweak constraints on WithCollectionBuilder so i can drop bad constructor
* Added SqliteDistributedLockingMechanism
* Dropped InMemoryDistributedMechanism + magic
InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.
Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.
However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.
* Fix quick install "" database name.
* Hide Database Configuration section when a connection string is pre-set.
Doesn't seem worth it to extract db name from connection string.
* Ensure wal test 2+
* Fix logging inconsistencies.
* Ensure in transaction when obtaining locks + no-op the SQLite read lock.
There's no point in running the query just to make a single test pass.
* Fix installer database display names
* Allow SQLite shared cache without losing deferred transactions
* Opt into shared cache for new SQLite databases + fix filename
* Fix misc inconsistency in .gitignore
* Prefer our interceptor interface
* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.
* Back to private cache.
* Added retry strategy for SQLite + refactor out SQL server specific stuff
* Fix SQL server tests.
* Misc - Orphaned comment, incorrect casing.
* InMemory SQLite test database & turn shared cache back on everywhere.
Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
2022-03-11 16:14:20 +00:00
|
|
|
using Umbraco.Cms.Persistence.Sqlite;
|
|
|
|
|
using Umbraco.Cms.Persistence.SqlServer;
|
2021-02-18 11:06:02 +01:00
|
|
|
using Umbraco.Cms.Tests.Common.Testing;
|
|
|
|
|
using Umbraco.Cms.Tests.Integration.DependencyInjection;
|
|
|
|
|
using Umbraco.Cms.Tests.Integration.Testing;
|
|
|
|
|
using Umbraco.Cms.Web.BackOffice.Controllers;
|
|
|
|
|
using Umbraco.Cms.Web.Common.Controllers;
|
|
|
|
|
using Umbraco.Cms.Web.Website.Controllers;
|
2020-06-17 16:39:28 +02:00
|
|
|
using Umbraco.Extensions;
|
|
|
|
|
|
2021-02-18 11:06:02 +01:00
|
|
|
namespace Umbraco.Cms.Tests.Integration.TestServerTest
|
2020-06-17 16:39:28 +02:00
|
|
|
{
|
|
|
|
|
[TestFixture]
|
2020-10-16 09:54:03 +02:00
|
|
|
[UmbracoTest(Database = UmbracoTestOptions.Database.NewSchemaPerTest, Logger = UmbracoTestOptions.Logger.Console, Boot = true)]
|
2022-02-11 20:37:56 +00:00
|
|
|
public abstract class UmbracoTestServerTestBase : UmbracoIntegrationTestBase
|
2020-06-17 16:39:28 +02:00
|
|
|
{
|
2022-02-11 20:37:56 +00:00
|
|
|
protected HttpClient Client { get; private set; }
|
|
|
|
|
|
|
|
|
|
protected LinkGenerator LinkGenerator { get; private set; }
|
|
|
|
|
|
|
|
|
|
protected WebApplicationFactory<UmbracoTestServerTestBase> Factory { get; private set; }
|
|
|
|
|
|
2020-06-17 16:39:28 +02:00
|
|
|
[SetUp]
|
2022-02-11 20:37:56 +00:00
|
|
|
public void Setup()
|
2020-06-17 16:39:28 +02:00
|
|
|
{
|
2021-09-29 11:51:28 +01:00
|
|
|
/*
|
|
|
|
|
* It's worth noting that our usage of WebApplicationFactory is non-standard,
|
|
|
|
|
* the intent is that your Startup.ConfigureServices is called just like
|
|
|
|
|
* when the app starts up, then replacements are registered in this class with
|
|
|
|
|
* builder.ConfigureServices (builder.ConfigureTestServices has hung around from before the
|
|
|
|
|
* generic host switchover).
|
|
|
|
|
*
|
|
|
|
|
* This is currently a pain to refactor towards due to UmbracoBuilder+TypeFinder+TypeLoader setup but
|
|
|
|
|
* we should get there one day.
|
|
|
|
|
*
|
2022-02-11 20:37:56 +00:00
|
|
|
* However we need to separate the testing framework we provide for downstream projects from our own tests.
|
|
|
|
|
* We cannot use the Umbraco.Web.UI startup yet as that is not available downstream.
|
|
|
|
|
*
|
2021-09-29 11:51:28 +01:00
|
|
|
* See https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests
|
|
|
|
|
*/
|
2022-02-11 20:37:56 +00:00
|
|
|
var factory = new UmbracoWebApplicationFactory<UmbracoTestServerTestBase>(CreateHostBuilder);
|
2020-09-02 18:10:29 +10:00
|
|
|
|
|
|
|
|
// additional host configuration for web server integration tests
|
|
|
|
|
Factory = factory.WithWebHostBuilder(builder =>
|
2021-10-19 10:09:48 +01:00
|
|
|
{
|
|
|
|
|
// Otherwise inferred as $(SolutionDir)/Umbraco.Tests.Integration (note lack of src/tests)
|
|
|
|
|
builder.UseContentRoot(Assembly.GetExecutingAssembly().GetRootDirectorySafe());
|
2020-12-23 11:35:49 +01:00
|
|
|
|
2020-09-02 18:10:29 +10:00
|
|
|
// Executes after the standard ConfigureServices method
|
|
|
|
|
builder.ConfigureTestServices(services =>
|
2020-12-23 11:35:49 +01:00
|
|
|
|
2020-11-26 16:52:03 +11:00
|
|
|
// Add a test auth scheme with a test auth handler to authn and assign the user
|
|
|
|
|
services.AddAuthentication(TestAuthHandler.TestAuthenticationScheme)
|
2021-10-19 10:09:48 +01:00
|
|
|
.AddScheme<AuthenticationSchemeOptions, TestAuthHandler>(TestAuthHandler.TestAuthenticationScheme, options => { }));
|
|
|
|
|
});
|
|
|
|
|
|
2020-09-02 18:10:29 +10:00
|
|
|
Client = Factory.CreateClient(new WebApplicationFactoryClientOptions
|
2020-09-04 00:27:43 +10:00
|
|
|
{
|
|
|
|
|
AllowAutoRedirect = false
|
2020-09-02 18:10:29 +10:00
|
|
|
});
|
|
|
|
|
|
2020-06-17 16:39:28 +02:00
|
|
|
LinkGenerator = Factory.Services.GetRequiredService<LinkGenerator>();
|
2020-08-31 11:31:56 +02:00
|
|
|
}
|
|
|
|
|
|
2020-07-06 12:55:23 +02:00
|
|
|
/// <summary>
|
|
|
|
|
/// Prepare a url before using <see cref="Client"/>.
|
|
|
|
|
/// This returns the url but also sets the HttpContext.request into to use this url.
|
|
|
|
|
/// </summary>
|
|
|
|
|
/// <returns>The string URL of the controller action.</returns>
|
2021-02-02 14:46:46 +11:00
|
|
|
protected string PrepareApiControllerUrl<T>(Expression<Func<T, object>> methodSelector)
|
2020-06-17 16:39:28 +02:00
|
|
|
where T : UmbracoApiController
|
|
|
|
|
{
|
2022-02-11 20:37:56 +00:00
|
|
|
var url = LinkGenerator.GetUmbracoApiService(methodSelector);
|
2021-02-02 14:46:46 +11:00
|
|
|
return PrepareUrl(url);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/// <summary>
|
|
|
|
|
/// Prepare a url before using <see cref="Client"/>.
|
|
|
|
|
/// This returns the url but also sets the HttpContext.request into to use this url.
|
|
|
|
|
/// </summary>
|
|
|
|
|
/// <returns>The string URL of the controller action.</returns>
|
|
|
|
|
protected string PrepareSurfaceControllerUrl<T>(Expression<Func<T, object>> methodSelector)
|
|
|
|
|
where T : SurfaceController
|
|
|
|
|
{
|
2022-02-11 20:37:56 +00:00
|
|
|
var url = LinkGenerator.GetUmbracoSurfaceUrl(methodSelector);
|
2021-02-02 14:46:46 +11:00
|
|
|
return PrepareUrl(url);
|
|
|
|
|
}
|
2020-06-17 16:39:28 +02:00
|
|
|
|
2021-02-02 14:46:46 +11:00
|
|
|
/// <summary>
|
|
|
|
|
/// Prepare a url before using <see cref="Client"/>.
|
|
|
|
|
/// This returns the url but also sets the HttpContext.request into to use this url.
|
|
|
|
|
/// </summary>
|
|
|
|
|
/// <returns>The string URL of the controller action.</returns>
|
|
|
|
|
protected string PrepareUrl(string url)
|
|
|
|
|
{
|
2020-12-23 11:35:49 +01:00
|
|
|
IUmbracoContextFactory umbracoContextFactory = GetRequiredService<IUmbracoContextFactory>();
|
|
|
|
|
IHttpContextAccessor httpContextAccessor = GetRequiredService<IHttpContextAccessor>();
|
2020-06-17 16:39:28 +02:00
|
|
|
|
|
|
|
|
httpContextAccessor.HttpContext = new DefaultHttpContext
|
|
|
|
|
{
|
|
|
|
|
Request =
|
|
|
|
|
{
|
|
|
|
|
Scheme = "https",
|
|
|
|
|
Host = new HostString("localhost", 80),
|
|
|
|
|
Path = url,
|
|
|
|
|
QueryString = new QueryString(string.Empty)
|
|
|
|
|
}
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
umbracoContextFactory.EnsureUmbracoContext();
|
|
|
|
|
|
|
|
|
|
return url;
|
|
|
|
|
}
|
|
|
|
|
|
2022-02-11 20:37:56 +00:00
|
|
|
private IHostBuilder CreateHostBuilder()
|
|
|
|
|
{
|
|
|
|
|
IHostBuilder hostBuilder = Host.CreateDefaultBuilder()
|
|
|
|
|
.ConfigureAppConfiguration((context, configBuilder) =>
|
|
|
|
|
{
|
|
|
|
|
context.HostingEnvironment = TestHelper.GetWebHostEnvironment();
|
|
|
|
|
configBuilder.Sources.Clear();
|
|
|
|
|
configBuilder.AddInMemoryCollection(InMemoryConfiguration);
|
v10 SQLite support + distributed locking abstractions (#11922)
* Created Persistence.SQLite project skeleton.
* SQLite database initialization
* Various changes and hacks to make things work.
* WIP integration tests
* Fix thread safety tests
* Fix tests that relied on tie breaker sorting.
Spent a fair amount of time looking for a less lazy fix but gave up.
* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join
* Fix test Can_Generate_Delete_SubQuery_Statement
Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].
SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."
Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.
* Fixes for paginated custom order by
* Fix tests broken by lack of unique indexes.
* Fix SqlServerTableByTableTest tests.
These tests didn't actually do anything as the tables already exist so schema creator just returned.
Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.
* Fix ValidateLoginSession - SelectTop must come later
* dry up database cleanup
* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.
* Prevent schema creation errors.
* SQLite ignore lock tests, WAL back on.
* Fix package schema tests
* Fix NPocoFetchTests - case sensitivity not under test
* Fix AdvancedMigrationTests (where possible)
Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.
If we do that can do some cleanup.
* Cleanup test database configuration
* Run integration tests against SQLite on build agent.
* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping
* YAML
* Skip Umbraco.Tests.Integration.SqlCe
* Drop SqlServerTableByTable tests.
Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).
Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors
* Might aswell run against macOS also.
* Copy azure pipelines task header layout
* Delete SQLCe projects
* Remove SQL CE specific code.
* Remove SQL CE NuSpec, template params, build script setup
* Delete umbraco-netcore-only.sln
* Add SkipTests solution configuration and use for codeql
* Remove reference to deleted nuspec file.
* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.
At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.
"ConnectionStrings": {
"umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
"umbracoDbDSN_ProviderName": "System.Data.SQLite"
},
Not currently possible via installer UI without provider name pre-set in
configuration.
* Switch to Microsoft.Data.Sqlite
Some gross hacks but will be good to find out if this works
with apple silicon.
* Enable selection of SQLite via installer UI (also quick install)
* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO
* Move SQL Server support to its own class library
* Add persistence dependencies to Umbraco.CMS metapackage
* Bugfix packages delete query
Created invalid query for SQLite.
* Try out cypress tests Linux + SQLite
* Prevent cypress test artifact upload failure on attempt 2+
* LocalDb bugfixes
* Drop redundant enum
* Move SqlClient constant
* Misc whitespace
* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).
* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*
* Display quick install defaults and per provider default database name.
* Misc remove old comment
* little re-arrange
* Remove almost all usages of IsSqlite extension.
* visual adjustments
* Custom Database Configuration is last step and should then say Install.
* use text instead of disabled inputs
* move legend, rename to Install
* Update SqlMainDomLock to work without distributed locks.
* Added IDistributedLockingMechanism interface and in memory impl.
* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.
* Added SqlServerDistributedLockingMechanism
* Move distributed locking interfaces and exceptions to Core + xmldocs.
* Fix tests, Misc cleanup, Add SQL distributed locking integration tests
* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)
* Nomplementation -> NoImplementation
* Fix misleading comment
* Integration tests use SqlServerDistributedLockingMechanism when possible
* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.
Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests
* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.
* Tweak constraints on WithCollectionBuilder so i can drop bad constructor
* Added SqliteDistributedLockingMechanism
* Dropped InMemoryDistributedMechanism + magic
InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.
Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.
However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.
* Fix quick install "" database name.
* Hide Database Configuration section when a connection string is pre-set.
Doesn't seem worth it to extract db name from connection string.
* Ensure wal test 2+
* Fix logging inconsistencies.
* Ensure in transaction when obtaining locks + no-op the SQLite read lock.
There's no point in running the query just to make a single test pass.
* Fix installer database display names
* Allow SQLite shared cache without losing deferred transactions
* Opt into shared cache for new SQLite databases + fix filename
* Fix misc inconsistency in .gitignore
* Prefer our interceptor interface
* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.
* Back to private cache.
* Added retry strategy for SQLite + refactor out SQL server specific stuff
* Fix SQL server tests.
* Misc - Orphaned comment, incorrect casing.
* InMemory SQLite test database & turn shared cache back on everywhere.
Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
2022-03-11 16:14:20 +00:00
|
|
|
configBuilder.AddConfiguration(GlobalSetupTeardown.TestConfiguration);
|
2020-06-17 16:39:28 +02:00
|
|
|
|
2022-02-11 20:37:56 +00:00
|
|
|
Configuration = configBuilder.Build();
|
|
|
|
|
})
|
|
|
|
|
.ConfigureWebHost(builder =>
|
|
|
|
|
{
|
2022-02-12 11:40:39 +00:00
|
|
|
builder.ConfigureServices((context, services) =>
|
|
|
|
|
{
|
|
|
|
|
context.HostingEnvironment = TestHelper.GetWebHostEnvironment();
|
|
|
|
|
|
|
|
|
|
ConfigureServices(services);
|
2022-02-12 11:57:28 +00:00
|
|
|
ConfigureTestServices(services);
|
2022-04-25 08:33:03 +01:00
|
|
|
services.AddUnique(CreateLoggerFactory());
|
2022-02-12 11:40:39 +00:00
|
|
|
|
|
|
|
|
if (!TestOptions.Boot)
|
|
|
|
|
{
|
|
|
|
|
// If boot is false, we don't want the CoreRuntime hosted service to start
|
|
|
|
|
// So we replace it with a Mock
|
|
|
|
|
services.AddUnique(Mock.Of<IRuntime>());
|
|
|
|
|
}
|
|
|
|
|
});
|
2020-09-02 18:10:29 +10:00
|
|
|
|
2022-02-11 20:37:56 +00:00
|
|
|
// call startup
|
|
|
|
|
builder.Configure(Configure);
|
|
|
|
|
})
|
|
|
|
|
.UseDefaultServiceProvider(cfg =>
|
|
|
|
|
{
|
|
|
|
|
// These default to true *if* WebHostEnvironment.EnvironmentName == Development
|
|
|
|
|
// When running tests, EnvironmentName used to be null on the mock that we register into services.
|
|
|
|
|
// Enable opt in for tests so that validation occurs regardless of environment name.
|
|
|
|
|
// Would be nice to have this on for UmbracoIntegrationTest also but requires a lot more effort to resolve issues.
|
|
|
|
|
cfg.ValidateOnBuild = true;
|
|
|
|
|
cfg.ValidateScopes = true;
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
return hostBuilder;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
protected virtual IServiceProvider Services => Factory.Services;
|
|
|
|
|
|
|
|
|
|
protected virtual T GetRequiredService<T>() => Factory.Services.GetRequiredService<T>();
|
2020-06-17 16:39:28 +02:00
|
|
|
|
2022-02-12 11:57:28 +00:00
|
|
|
protected void ConfigureServices(IServiceCollection services)
|
2020-09-02 18:10:29 +10:00
|
|
|
{
|
2020-12-11 18:20:07 +00:00
|
|
|
services.AddTransient<TestUmbracoDatabaseFactoryProvider>();
|
2021-08-19 09:08:12 +02:00
|
|
|
|
|
|
|
|
Core.Hosting.IHostingEnvironment hostingEnvironment = TestHelper.GetHostingEnvironment();
|
2022-02-11 20:37:56 +00:00
|
|
|
|
2020-12-23 11:35:49 +01:00
|
|
|
TypeLoader typeLoader = services.AddTypeLoader(
|
2020-12-21 15:58:47 +11:00
|
|
|
GetType().Assembly,
|
2021-08-19 09:08:12 +02:00
|
|
|
hostingEnvironment,
|
2020-12-21 15:58:47 +11:00
|
|
|
TestHelper.ConsoleLoggerFactory,
|
|
|
|
|
AppCaches.NoCache,
|
|
|
|
|
Configuration,
|
|
|
|
|
TestHelper.Profiler);
|
2020-11-20 12:24:16 +00:00
|
|
|
|
2021-08-19 09:08:12 +02:00
|
|
|
var builder = new UmbracoBuilder(services, Configuration, typeLoader, TestHelper.ConsoleLoggerFactory, TestHelper.Profiler, AppCaches.NoCache, hostingEnvironment);
|
2020-12-11 18:20:07 +00:00
|
|
|
|
2020-11-20 12:24:16 +00:00
|
|
|
builder
|
2020-11-19 09:06:04 +00:00
|
|
|
.AddConfiguration()
|
2020-12-24 16:35:59 +11:00
|
|
|
.AddUmbracoCore()
|
2020-11-19 09:06:04 +00:00
|
|
|
.AddWebComponents()
|
Examine 2.0 integration (#10241)
* Init commit for examine 2.0 work, most old umb examine tests working, probably a lot that doesn't
* Gets Umbraco Examine tests passing and makes some sense out of them, fixes some underlying issues.
* Large refactor, remove TaskHelper, rename Notifications to be consistent, Gets all examine/lucene indexes building and startup ordered in the correct way, removes old files, creates new IUmbracoIndexingHandler for abstracting out all index operations for umbraco data, abstracts out IIndexRebuilder, Fixes Stack overflow with LiveModelsProvider and loading assemblies, ports some changes from v8 for startup handling with cold boots, refactors out LastSyncedFileManager
* fix up issues with rebuilding and management dashboard.
* removes old files, removes NetworkHelper, fixes LastSyncedFileManager implementation to ensure the machine name is used, fix up logging with cold boot state.
* Makes MainDom safer to use and makes PublishedSnapshotService lazily register with MainDom
* lazily acquire application id (fix unit tests)
* Fixes resource casing and missing test file
* Ensures caches when requiring internal services for PublishedSnapshotService, UseNuCache is a separate call, shouldn't be buried in AddWebComponents, was also causing issues in integration tests since nucache was being used for the Id2Key service.
* For UmbracoTestServerTestBase enable nucache services
* Fixing tests
* Fix another test
* Fixes tests, use TestHostingEnvironment, make Tests.Common use net5, remove old Lucene.Net.Contrib ref.
* Fixes up some review notes
* Fixes issue with doubly registering PublishedSnapshotService meanig there could be 2x instances of it
* Checks for parseexception when executing the query
* Use application root instead of duplicating functionality.
* Added Examine project to netcore only solution file
* Fixed casing issue with LazyLoad, that is not lowercase.
* uses cancellationToken instead of bool flag, fixes always reading lastId from the LastSyncedFileManager, fixes RecurringHostedServiceBase so that there isn't an overlapping thread for the same task type
* Fix tests
* remove legacy test project from solution file
* Fix test
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-05-18 18:31:38 +10:00
|
|
|
.AddNuCache()
|
2021-03-16 19:19:03 +11:00
|
|
|
.AddRuntimeMinifier()
|
2020-12-24 18:11:16 +11:00
|
|
|
.AddBackOfficeCore()
|
2020-12-22 12:16:37 +11:00
|
|
|
.AddBackOfficeAuthentication()
|
2020-11-19 09:06:04 +00:00
|
|
|
.AddBackOfficeIdentity()
|
2020-12-08 11:25:01 +00:00
|
|
|
.AddMembersIdentity()
|
2020-11-26 16:52:03 +11:00
|
|
|
.AddBackOfficeAuthorizationPolicies(TestAuthHandler.TestAuthenticationScheme)
|
2020-11-19 09:06:04 +00:00
|
|
|
.AddPreviewSupport()
|
|
|
|
|
.AddMvcAndRazor(mvcBuilding: mvcBuilder =>
|
2020-09-02 18:10:29 +10:00
|
|
|
{
|
2020-12-21 15:58:47 +11:00
|
|
|
// Adds Umbraco.Web.BackOffice
|
2020-09-04 00:27:43 +10:00
|
|
|
mvcBuilder.AddApplicationPart(typeof(ContentController).Assembly);
|
2020-12-21 15:58:47 +11:00
|
|
|
|
|
|
|
|
// Adds Umbraco.Web.Common
|
|
|
|
|
mvcBuilder.AddApplicationPart(typeof(RenderController).Assembly);
|
|
|
|
|
|
|
|
|
|
// Adds Umbraco.Web.Website
|
|
|
|
|
mvcBuilder.AddApplicationPart(typeof(SurfaceController).Assembly);
|
2021-02-02 14:46:46 +11:00
|
|
|
|
|
|
|
|
// Adds Umbraco.Tests.Integration
|
|
|
|
|
mvcBuilder.AddApplicationPart(typeof(UmbracoTestServerTestBase).Assembly);
|
2020-09-02 18:10:29 +10:00
|
|
|
})
|
2020-11-19 09:06:04 +00:00
|
|
|
.AddWebServer()
|
2021-01-04 11:32:36 +11:00
|
|
|
.AddWebsite()
|
v10 SQLite support + distributed locking abstractions (#11922)
* Created Persistence.SQLite project skeleton.
* SQLite database initialization
* Various changes and hacks to make things work.
* WIP integration tests
* Fix thread safety tests
* Fix tests that relied on tie breaker sorting.
Spent a fair amount of time looking for a less lazy fix but gave up.
* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join
* Fix test Can_Generate_Delete_SubQuery_Statement
Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].
SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."
Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.
* Fixes for paginated custom order by
* Fix tests broken by lack of unique indexes.
* Fix SqlServerTableByTableTest tests.
These tests didn't actually do anything as the tables already exist so schema creator just returned.
Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.
* Fix ValidateLoginSession - SelectTop must come later
* dry up database cleanup
* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.
* Prevent schema creation errors.
* SQLite ignore lock tests, WAL back on.
* Fix package schema tests
* Fix NPocoFetchTests - case sensitivity not under test
* Fix AdvancedMigrationTests (where possible)
Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.
If we do that can do some cleanup.
* Cleanup test database configuration
* Run integration tests against SQLite on build agent.
* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping
* YAML
* Skip Umbraco.Tests.Integration.SqlCe
* Drop SqlServerTableByTable tests.
Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).
Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors
* Might aswell run against macOS also.
* Copy azure pipelines task header layout
* Delete SQLCe projects
* Remove SQL CE specific code.
* Remove SQL CE NuSpec, template params, build script setup
* Delete umbraco-netcore-only.sln
* Add SkipTests solution configuration and use for codeql
* Remove reference to deleted nuspec file.
* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.
At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.
"ConnectionStrings": {
"umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
"umbracoDbDSN_ProviderName": "System.Data.SQLite"
},
Not currently possible via installer UI without provider name pre-set in
configuration.
* Switch to Microsoft.Data.Sqlite
Some gross hacks but will be good to find out if this works
with apple silicon.
* Enable selection of SQLite via installer UI (also quick install)
* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO
* Move SQL Server support to its own class library
* Add persistence dependencies to Umbraco.CMS metapackage
* Bugfix packages delete query
Created invalid query for SQLite.
* Try out cypress tests Linux + SQLite
* Prevent cypress test artifact upload failure on attempt 2+
* LocalDb bugfixes
* Drop redundant enum
* Move SqlClient constant
* Misc whitespace
* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).
* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*
* Display quick install defaults and per provider default database name.
* Misc remove old comment
* little re-arrange
* Remove almost all usages of IsSqlite extension.
* visual adjustments
* Custom Database Configuration is last step and should then say Install.
* use text instead of disabled inputs
* move legend, rename to Install
* Update SqlMainDomLock to work without distributed locks.
* Added IDistributedLockingMechanism interface and in memory impl.
* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.
* Added SqlServerDistributedLockingMechanism
* Move distributed locking interfaces and exceptions to Core + xmldocs.
* Fix tests, Misc cleanup, Add SQL distributed locking integration tests
* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)
* Nomplementation -> NoImplementation
* Fix misleading comment
* Integration tests use SqlServerDistributedLockingMechanism when possible
* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.
Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests
* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.
* Tweak constraints on WithCollectionBuilder so i can drop bad constructor
* Added SqliteDistributedLockingMechanism
* Dropped InMemoryDistributedMechanism + magic
InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.
Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.
However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.
* Fix quick install "" database name.
* Hide Database Configuration section when a connection string is pre-set.
Doesn't seem worth it to extract db name from connection string.
* Ensure wal test 2+
* Fix logging inconsistencies.
* Ensure in transaction when obtaining locks + no-op the SQLite read lock.
There's no point in running the query just to make a single test pass.
* Fix installer database display names
* Allow SQLite shared cache without losing deferred transactions
* Opt into shared cache for new SQLite databases + fix filename
* Fix misc inconsistency in .gitignore
* Prefer our interceptor interface
* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.
* Back to private cache.
* Added retry strategy for SQLite + refactor out SQL server specific stuff
* Fix SQL server tests.
* Misc - Orphaned comment, incorrect casing.
* InMemory SQLite test database & turn shared cache back on everywhere.
Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
2022-03-11 16:14:20 +00:00
|
|
|
.AddUmbracoSqlServerSupport()
|
|
|
|
|
.AddUmbracoSqliteSupport()
|
2020-12-24 16:35:59 +11:00
|
|
|
.AddTestServices(TestHelper) // This is the important one!
|
2020-09-02 18:10:29 +10:00
|
|
|
.Build();
|
|
|
|
|
}
|
|
|
|
|
|
2022-02-12 11:57:28 +00:00
|
|
|
/// <summary>
|
|
|
|
|
/// Hook for registering test doubles.
|
|
|
|
|
/// </summary>
|
|
|
|
|
protected virtual void ConfigureTestServices(IServiceCollection services)
|
|
|
|
|
{
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
protected void Configure(IApplicationBuilder app)
|
2020-09-04 00:27:43 +10:00
|
|
|
{
|
2022-02-11 20:37:56 +00:00
|
|
|
UseTestDatabase(app);
|
|
|
|
|
|
Implements Public Access in netcore (#10137)
* Getting new netcore PublicAccessChecker in place
* Adds full test coverage for PublicAccessChecker
* remove PublicAccessComposer
* adjust namespaces, ensure RoleManager works, separate public access controller, reduce content controller
* Implements the required methods on IMemberManager, removes old migrated code
* Updates routing to be able to re-route, Fixes middleware ordering ensuring endpoints are last, refactors pipeline options, adds public access middleware, ensures public access follows all hops
* adds note
* adds note
* Cleans up ext methods, ensures that members identity is added on both front-end and back ends. updates how UmbracoApplicationBuilder works in that it explicitly starts endpoints at the time of calling.
* Changes name to IUmbracoEndpointBuilder
* adds note
* Fixing tests, fixing error describers so there's 2x one for back office, one for members, fixes TryConvertTo, fixes login redirect
* fixing build
* Fixes keepalive, fixes PublicAccessMiddleware to not throw, updates startup code to be more clear and removes magic that registers middleware.
* adds note
* removes unused filter, fixes build
* fixes WebPath and tests
* Looks up entities in one query
* remove usings
* Fix test, remove stylesheet
* Set status code before we write to response to avoid error
* Ensures that users and members are validated when logging in. Shares more code between users and members.
* Fixes RepositoryCacheKeys to ensure the keys are normalized
* oops didn't mean to commit this
* Fix casing issues with caching, stop boxing value types for all cache operations, stop re-creating string keys in DefaultRepositoryCachePolicy
* bah, far out this keeps getting recommitted. sorry
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-04-20 15:11:45 +10:00
|
|
|
app.UseUmbraco()
|
Implement password config storage for members (#10170)
* Getting new netcore PublicAccessChecker in place
* Adds full test coverage for PublicAccessChecker
* remove PublicAccessComposer
* adjust namespaces, ensure RoleManager works, separate public access controller, reduce content controller
* Implements the required methods on IMemberManager, removes old migrated code
* Updates routing to be able to re-route, Fixes middleware ordering ensuring endpoints are last, refactors pipeline options, adds public access middleware, ensures public access follows all hops
* adds note
* adds note
* Cleans up ext methods, ensures that members identity is added on both front-end and back ends. updates how UmbracoApplicationBuilder works in that it explicitly starts endpoints at the time of calling.
* Changes name to IUmbracoEndpointBuilder
* adds note
* Fixing tests, fixing error describers so there's 2x one for back office, one for members, fixes TryConvertTo, fixes login redirect
* fixing build
* Updates user manager to correctly validate password hashing and injects the IBackOfficeUserPasswordChecker
* Merges PR
* Fixes up build and notes
* Implements security stamp and email confirmed for members, cleans up a bunch of repo/service level member groups stuff, shares user store code between members and users and fixes the user identity object so we arent' tracking both groups and roles.
* Security stamp for members is now working
* Fixes keepalive, fixes PublicAccessMiddleware to not throw, updates startup code to be more clear and removes magic that registers middleware.
* adds note
* removes unused filter, fixes build
* fixes WebPath and tests
* Looks up entities in one query
* remove usings
* Fix test, remove stylesheet
* Set status code before we write to response to avoid error
* Ensures that users and members are validated when logging in. Shares more code between users and members.
* merge changes
* oops
* Reducing and removing published member cache
* Fixes RepositoryCacheKeys to ensure the keys are normalized
* oops didn't mean to commit this
* Fix casing issues with caching, stop boxing value types for all cache operations, stop re-creating string keys in DefaultRepositoryCachePolicy
* oops didn't mean to comit this
* bah, far out this keeps getting recommitted. sorry
* cannot inject IPublishedMemberCache and cannot have IPublishedMember
* splits out files, fixes build
* fix tests
* removes membership provider classes
* removes membership provider classes
* updates the identity map definition
* reverts commented out lines
* reverts commented out lines
* Implements members Password config in db, fixes members cookie auth to not interfere with the back office cookie auth, fixes Startup sequence, fixes startup pipeline
* commits change to Startup
* Rename migration from `MemberTableColumns2` to `AddPasswordConfigToMemberTable`
* Fix test
* Fix tests, but adding default passwordConfig to members
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-04-22 23:59:13 +10:00
|
|
|
.WithMiddleware(u =>
|
|
|
|
|
{
|
2021-07-16 16:04:14 -06:00
|
|
|
u.UseBackOffice();
|
|
|
|
|
u.UseWebsite();
|
Implement password config storage for members (#10170)
* Getting new netcore PublicAccessChecker in place
* Adds full test coverage for PublicAccessChecker
* remove PublicAccessComposer
* adjust namespaces, ensure RoleManager works, separate public access controller, reduce content controller
* Implements the required methods on IMemberManager, removes old migrated code
* Updates routing to be able to re-route, Fixes middleware ordering ensuring endpoints are last, refactors pipeline options, adds public access middleware, ensures public access follows all hops
* adds note
* adds note
* Cleans up ext methods, ensures that members identity is added on both front-end and back ends. updates how UmbracoApplicationBuilder works in that it explicitly starts endpoints at the time of calling.
* Changes name to IUmbracoEndpointBuilder
* adds note
* Fixing tests, fixing error describers so there's 2x one for back office, one for members, fixes TryConvertTo, fixes login redirect
* fixing build
* Updates user manager to correctly validate password hashing and injects the IBackOfficeUserPasswordChecker
* Merges PR
* Fixes up build and notes
* Implements security stamp and email confirmed for members, cleans up a bunch of repo/service level member groups stuff, shares user store code between members and users and fixes the user identity object so we arent' tracking both groups and roles.
* Security stamp for members is now working
* Fixes keepalive, fixes PublicAccessMiddleware to not throw, updates startup code to be more clear and removes magic that registers middleware.
* adds note
* removes unused filter, fixes build
* fixes WebPath and tests
* Looks up entities in one query
* remove usings
* Fix test, remove stylesheet
* Set status code before we write to response to avoid error
* Ensures that users and members are validated when logging in. Shares more code between users and members.
* merge changes
* oops
* Reducing and removing published member cache
* Fixes RepositoryCacheKeys to ensure the keys are normalized
* oops didn't mean to commit this
* Fix casing issues with caching, stop boxing value types for all cache operations, stop re-creating string keys in DefaultRepositoryCachePolicy
* oops didn't mean to comit this
* bah, far out this keeps getting recommitted. sorry
* cannot inject IPublishedMemberCache and cannot have IPublishedMember
* splits out files, fixes build
* fix tests
* removes membership provider classes
* removes membership provider classes
* updates the identity map definition
* reverts commented out lines
* reverts commented out lines
* Implements members Password config in db, fixes members cookie auth to not interfere with the back office cookie auth, fixes Startup sequence, fixes startup pipeline
* commits change to Startup
* Rename migration from `MemberTableColumns2` to `AddPasswordConfigToMemberTable`
* Fix test
* Fix tests, but adding default passwordConfig to members
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-04-22 23:59:13 +10:00
|
|
|
})
|
Implements Public Access in netcore (#10137)
* Getting new netcore PublicAccessChecker in place
* Adds full test coverage for PublicAccessChecker
* remove PublicAccessComposer
* adjust namespaces, ensure RoleManager works, separate public access controller, reduce content controller
* Implements the required methods on IMemberManager, removes old migrated code
* Updates routing to be able to re-route, Fixes middleware ordering ensuring endpoints are last, refactors pipeline options, adds public access middleware, ensures public access follows all hops
* adds note
* adds note
* Cleans up ext methods, ensures that members identity is added on both front-end and back ends. updates how UmbracoApplicationBuilder works in that it explicitly starts endpoints at the time of calling.
* Changes name to IUmbracoEndpointBuilder
* adds note
* Fixing tests, fixing error describers so there's 2x one for back office, one for members, fixes TryConvertTo, fixes login redirect
* fixing build
* Fixes keepalive, fixes PublicAccessMiddleware to not throw, updates startup code to be more clear and removes magic that registers middleware.
* adds note
* removes unused filter, fixes build
* fixes WebPath and tests
* Looks up entities in one query
* remove usings
* Fix test, remove stylesheet
* Set status code before we write to response to avoid error
* Ensures that users and members are validated when logging in. Shares more code between users and members.
* Fixes RepositoryCacheKeys to ensure the keys are normalized
* oops didn't mean to commit this
* Fix casing issues with caching, stop boxing value types for all cache operations, stop re-creating string keys in DefaultRepositoryCachePolicy
* bah, far out this keeps getting recommitted. sorry
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-04-20 15:11:45 +10:00
|
|
|
.WithEndpoints(u =>
|
|
|
|
|
{
|
|
|
|
|
u.UseBackOfficeEndpoints();
|
|
|
|
|
u.UseWebsiteEndpoints();
|
|
|
|
|
});
|
2020-06-17 16:39:28 +02:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|