yv01p/Umbraco-CMS
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge in #9859 to v8/feature/nucache-perf (#10152)

Browse Source 
* load only once

* Bump version to 8.6.8

* Initial rework of Lock dictionaries

* [Issue 5277-146] accessibility - Close 'X' icon next to language drop… (#9264)

* [Issue 5277-146] accessibility - Close 'X' icon next to language drop down is identified as "link" - screen reader

* add new loacalization key

* Fix issue with SqlMainDomLock that cannot use implicit lock timeouts … (#9973)

* Fix issue with SqlMainDomLock that cannot use implicit lock timeouts … (#9973)

(cherry picked from commit da5351dfcf)

* Adjust unit tests and apply fixes to scope

* Add more unit tests, showing current issue

* Counting Umbraco.ModelsBuilder and ModelsBuilder.Umbraco namespaces as external providers

* Fix dead lock with TypeLoader

* Fix errors shown in unit tests

* Throw error if all scopes hasn't been disposed

* Clean

* Fixes and Updates for DB Scope and Ambient Context leaks (#9953)

* Adds some scope tests (ported back from netcore) and provides a much better error message, ensure execution context is not flowed to child tasks that shouldn't leak any current ambient context

* updates comment

* Ensure SqlMainDomLock suppresses execution context too

* Since we're awaiting a task in a library method, ConfigureAwait(false)

* missing null check

Co-authored-by: Elitsa Marinovska <elm@umbraco.dk>

* Adds additional error checking and reporting to MainDom/SqlMainDomLock (#9954)

Co-authored-by: Elitsa Marinovska <elm@umbraco.dk>

* Add copy logic to Media Picker (#9957)

* Add copy logic to Media Picker

* Add action for copy all

* Fix for selectable media item

* Wrap calls to map in scopes

* Autocomplete scopes

* Remove unnecessary aria-hidden attribute from <umb-icon>

* Remove scope from method that calls another method that has a scope

* Fixes #9993 - Cannot save empty image in Grid

* Clean

* Revert "The Value() method for IPublishedContent was not working with the defaultValue parameter" (#9989)

* Use a hashset to keep track of acquired locks

This simplifies disposing/checking for locks greatly.

* Add images in grid - fixes 9982 (#9987)

Co-authored-by: Sebastiaan Janssen <sebastiaan@umbraco.com>

* Only create the dicts and hashset when a lock is requested

* Clean

* Adds a config for configuring the access rules on the content dashboard - by default it granted for all user groups

* Adds additional params indicating whether user is admin

* Add images in grid - fixes 9982 (#9987)

Co-authored-by: Sebastiaan Janssen <sebastiaan@umbraco.com>
(cherry picked from commit e2019777fb)

* Bump version to 8.12.2

* #9964 Removed unneeded check for HttpContext

* Fix for #9950 - HttpsCheck will now retry using the login background image if inital request returns 301/302. Excessvie Headers check will now check the root url instead of the backoffice

* Merge pull request #9994 from umbraco/v8/bugfix/9993

Fixes #9993 - Cannot save empty image in Grid

(cherry picked from commit 0ecc933921)

* Apply suggestions from review

* Fixes #9983 - Getting kicked, if document type has a Umbraco.UserPicker property (#10002)

* Fixes #9983

Temporary fix for this issue. using the entityservice like before.

* Needed to remove the call to usersResource here as well for displaying the picked items

* Don't need usersResource for now

* Fixes #9983 - Getting kicked, if document type has a Umbraco.UserPicker property (#10002)

* Fixes #9983

Temporary fix for this issue. using the entityservice like before.

* Needed to remove the call to usersResource here as well for displaying the picked items

* Don't need usersResource for now

(cherry picked from commit 45de0a101e)

* 8539: Allow alias in image cropper (#9266)

Co-authored-by: Owain Williams <owaingdwilliams@gmail.com>

* Wrap dumping dictionaries in a method.

* Create method for generating log message

And remove forgotten comments.

* Fix swedish translation for somethingElse.

* Copy member type (#10020)

* Add copy dialog for member type

* Implement copy action for member type

* Create specific localization for content type, media type and member type

* Handle "foldersonly" querystring

* Add button type attribute

* Add a few missing changes of anchor to button element

* Null check on scope and options to ensure backward compatibility

* Improve performance, readability and handling of FollowInternalRedirects (#9889)

* Improve performance, readability and handling of FollowInternalRedirects

* Logger didn't like string param

Passing string param to _logger.Debug<PublishedRouter, int> wasn't happy. Changed to pass existing internalRedirectAsInt variable.

Co-authored-by: Nathan Woulfe <nathan@nathanw.com.au>

* Update casing of listview layout name

* 9097 add contextual password helper (#9256)

* update back-office forms

* Display tip on reset password page as well

* add directive for password tip

* integrate directove in login screen

* forgot the ng-keyup :-)

* adapt tooltip directive to potential different Members and Users password settings

* remove watcher

Co-authored-by: Nathan Woulfe <nathan@nathanw.com.au>

* Unbind listener

Listening for splitViewRequest was only unbound if the split view editor was opened. Not cleaning up the listener caused a memory leak when changing between nodes as the spit view editor was detached but not garbage-collected

* Replace icon in date picker with umb-icon component (#10040)

* Replace icon in date picker with <umb-icon> component

* Adjust height of clear button

* Update cypress and fix tests

* Listview config icons (#10036)

* Update icons to use <umb-icon> component

* Simplify markup and use disabled button

* Use move cursor style on sortable handle

* Add class for action column

* Update setting auto focus

* Increase font size of umb-panel-header-icon

* Anchor noopener (#10009)

* Set rel="noopener" for anchors with target="_blank"

* Reverted unwanted changes to Default.cshtml

* Align 'Add language' test to netcore

* Add new cypress tests

* Add indentation

* Getting rid of the config file and implementing an appSetting instead

* Implementation for IContentDashboardSettings

* Cleanup

* bool.Try

* Taking AllowContentDashboardAccessToAllUsers prop from GlobalSettings to ContentDashboardSettings and saving AccessRulesFromConfig into a backing field

* fix support for non run states

* Handling multiple values per field in Examine Management

* Add Root<T> and Breadcrumbs extension methods for IPublishedContent (#9033)

* Fix usage of obsolete CreatorName and WriterName properties

* Add generic Root extension method

* Add Breadcrumbs extension methods

* Orders member type grouping of members alphabetically, matching the listing of member types.

* Revert updating deprecated WriterName/CreatorName refs

Changing the properties to use the extensions is a good thing (given the props are deprecated), but causes issues deep in tests. I'm reverting that change to fix the tests, and all refs to the deprecated properties should be updated in one sweep, to deal with any other test issues that might crop up.

* Handle Invalid format for Upgrade check

* Fixes tabbing-mode remains active after closing modal #9790 (#10074)

* Allow to pass in boolean to preventEnterSubmit directive (#8639)

* Pass in value to preventEnterSubmit directive

* Set enabled similar to preventDefault and preventEnterSubmit directives

* Update prevent enter submit value

* Init value from controller

* Use a different default input id prefix for umb-search-filter

* Fix typo

* Check for truthly value

* Revert "Set enabled similar to preventDefault and preventEnterSubmit directives"

This reverts commit 536ce855c4545ead82cea77b4013bf9010a8687b.

* None pointer events when clicking icon

* Use color variable

* Fixes tabbing-mode remains active after closing modal #9790 (#10074)

(cherry picked from commit c881fa9e7d)

* Null check on scope and options to ensure backward compatibility

(cherry picked from commit fe8cd239d2)

* Fix validation of step size in integer/numeric field

* 9962: Use $allowedEditors instead of allowed (#10086)

* 9962: Use $allowedEditors instead of allowed

* 9962: Remove redundant statement

* fixes #10021 adds ng-form and val-form-manager to the documentation

* Improved accessibility of link picker (#10099)

* Added support for screeen reader alerts on the embed so that assitive technology knows when a url retrieve has been succesfull.
Added labels for the controls
Preview reload only triggered if the values for height and width change

* Added control ids for the link picker

* Add French translation

* Accessibility: Alerts the user how many results have been returned on a tree search (#10100)

* Added support for screeen reader alerts on the embed so that assitive technology knows when a url retrieve has been succesfull.
Added labels for the controls
Preview reload only triggered if the values for height and width change

* Tree search details the number of search items returned

* Add French translations

* Updated LightInject to v6.4.0

* Remove HtmlSanitizer once more - see #9803

* Also make sure NuGet installs the correct version of the CodePages dependency

* Bump version to 8.13 RC

* Fixed copy preserving sort order (#10091)

* Revert "Updated LightInject to v6.4.0"

This reverts commit fc77252ec7.

* Revert "Add copy logic to Media Picker (#9957)"

This reverts commit f7c032af65.

* Reintroduce old constructor to make non-breaking

* Update cypress test to make macros in the grid work again

* Attributes could be multiple items, test specifically if `Directory` is an attribute

* Accessibility: Adding label fors and control ids for the macro picker (#10101)

* Added support for screeen reader alerts on the embed so that assitive technology knows when a url retrieve has been succesfull.
Added labels for the controls
Preview reload only triggered if the values for height and width change

* Added support for label fors for the macro picker and also gave the ,acro search box a title

* Now displays a count of the matching macros returned. Please note the language file amends shared with #10100

* Removed src-only class for the display of the count of messages

* Updating typo

* Removed top-margin from switcher icon

* Allow KeepAlive controller Ping method to be requested by non local requests (#10126)

* Allow KeepAlive controller Ping method to be requested by non local requests and accept head requests

* removed unused references

* fix csproj

* fix merge

* btree serializer optimizations

* array pool and nametable optimizations

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>
Co-authored-by: Sebastiaan Janssen <sebastiaan@umbraco.com>
Co-authored-by: Justin Shearer <justin.shearer@gmail.com>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
Co-authored-by: Callum Whyte <hey@callumwhyte.com>
Co-authored-by: Shannon <sdeminick@gmail.com>
Co-authored-by: Elitsa Marinovska <elm@umbraco.dk>
Co-authored-by: patrickdemooij9 <patrickdemooij98@hotmail.com>
Co-authored-by: Bjarne Fyrstenborg <bjarne_fyrstenborg@hotmail.com>
Co-authored-by: Michael Latouche <michael@crossingpaths.be>
Co-authored-by: Nathan Woulfe <nathan@nathanw.com.au>
Co-authored-by: Markus Johansson <markus@obviuse.se>
Co-authored-by: Jeavon Leopold <jeavon@crumpled-dog.com>
Co-authored-by: Benjamin Carleski <benjamin@proworks.com>
Co-authored-by: Owain Williams <owaingdwilliams@gmail.com>
Co-authored-by: Jesper Löfgren <jesper.lofgren@sappa.se>
Co-authored-by: Martin Bentancour <mbe@umbraco.dk>
Co-authored-by: Ronald Barendse <ronald@panoramastudios.nl>
Co-authored-by: Andy Butland <abutland73@gmail.com>
Co-authored-by: BeardinaSuit <BeardInASuit@outlook.com>
Co-authored-by: Mads Rasmussen <madsr@hey.com>
Co-authored-by: Rachel Breeze <RachBreeze@users.noreply.github.com>
Co-authored-by: Dave de Moel <d.demoel@wearetriple.com>
Co-authored-by: ric <60885685+ricbrady@users.noreply.github.com>
Co-authored-by: Carole Rennie Logan <crgrieve@users.noreply.github.com>
Co-authored-by: Dennis Öhman <dennis.ohman@bjornmamman.se>
This commit is contained in:
Chad
2021-04-21 11:05:37 +12:00
committed by GitHub
parent 98d8971ce9
commit 7704d02883
16 changed files with 233 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

90
src/Umbraco.Core/Sync/DatabaseServerMessenger.cs
View File

@@ -28,7 +28,7 @@ namespace Umbraco.Core.Sync
// but only processes instructions coming from remote servers,
// thus ensuring that instructions run only once
//
public class DatabaseServerMessenger : ServerMessengerBase
public class DatabaseServerMessenger : ServerMessengerBase, ISyncBootStateAccessor
{
private readonly IRuntimeState _runtime;
private readonly ManualResetEvent _syncIdle;
@@ -172,35 +172,7 @@ namespace Umbraco.Core.Sync
lock (_locko)
{
if (_released) return;
var coldboot = false;
if (_lastId < 0) // never synced before
{
// we haven't synced - in this case we aren't going to sync the whole thing, we will assume this is a new
// server and it will need to rebuild it's own caches, eg Lucene or the xml cache file.
Logger.Warn<DatabaseServerMessenger>("No last synced Id found, this generally means this is a new server/install."
+ " The server will build its caches and indexes, and then adjust its last synced Id to the latest found in"
+ " the database and maintain cache updates based on that Id.");
coldboot = true;
}
else
{
//check for how many instructions there are to process, each row contains a count of the number of instructions contained in each
//row so we will sum these numbers to get the actual count.
var count = database.ExecuteScalar<int>("SELECT SUM(instructionCount) FROM umbracoCacheInstruction WHERE id > @lastId", new {lastId = _lastId});
if (count > Options.MaxProcessingInstructionCount)
{
//too many instructions, proceed to cold boot
Logger.Warn<DatabaseServerMessenger,int,int>(
"The instruction count ({InstructionCount}) exceeds the specified MaxProcessingInstructionCount ({MaxProcessingInstructionCount})."
+ " The server will skip existing instructions, rebuild its caches and indexes entirely, adjust its last synced Id"
+ " to the latest found in the database and maintain cache updates based on that Id.",
count, Options.MaxProcessingInstructionCount);
coldboot = true;
}
}
var coldboot = IsColdBoot(database);
if (coldboot)
{
@@ -223,6 +195,40 @@ namespace Umbraco.Core.Sync
}
}
private bool IsColdBoot(IUmbracoDatabase database)
{
var coldboot = false;
if (_lastId < 0) // never synced before
{
// we haven't synced - in this case we aren't going to sync the whole thing, we will assume this is a new
// server and it will need to rebuild it's own caches, eg Lucene or the xml cache file.
Logger.Warn<DatabaseServerMessenger>("No last synced Id found, this generally means this is a new server/install."
+ " The server will build its caches and indexes, and then adjust its last synced Id to the latest found in"
+ " the database and maintain cache updates based on that Id.");
coldboot = true;
}
else
{
//check for how many instructions there are to process, each row contains a count of the number of instructions contained in each
//row so we will sum these numbers to get the actual count.
var count = database.ExecuteScalar<int>("SELECT SUM(instructionCount) FROM umbracoCacheInstruction WHERE id > @lastId", new {lastId = _lastId});
if (count > Options.MaxProcessingInstructionCount)
{
//too many instructions, proceed to cold boot
Logger.Warn<DatabaseServerMessenger,int,int>(
"The instruction count ({InstructionCount}) exceeds the specified MaxProcessingInstructionCount ({MaxProcessingInstructionCount})."
+ " The server will skip existing instructions, rebuild its caches and indexes entirely, adjust its last synced Id"
+ " to the latest found in the database and maintain cache updates based on that Id.",
count, Options.MaxProcessingInstructionCount);
coldboot = true;
}
}
return coldboot;
}
/// <summary>
/// Synchronize the server (throttled).
/// </summary>
@@ -548,6 +554,30 @@ namespace Umbraco.Core.Sync
#endregion
public SyncBootState GetSyncBootState()
{
try
{
ReadLastSynced(); // get _lastId
using (var scope = ScopeProvider.CreateScope())
{
EnsureInstructions(scope.Database);
bool isColdBoot = IsColdBoot(scope.Database);
if (isColdBoot)
{
return SyncBootState.ColdBoot;
}
return SyncBootState.HasSyncState;
}
}
catch(Exception ex)
{
Logger.Warn<DatabaseServerMessenger>("Error determining Sync Boot State", ex);
return SyncBootState.Unknown;
}
}
#region Notify refreshers
private static ICacheRefresher GetRefresher(Guid id)

20
src/Umbraco.Core/Sync/ISyncBootStateAccessor.cs Normal file
View File

@@ -0,0 +1,20 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Umbraco.Core.Sync
{
/// <summary>
/// Retrieve the state of the sync service
/// </summary>
public interface ISyncBootStateAccessor
{
/// <summary>
/// Get the boot state
/// </summary>
/// <returns></returns>
SyncBootState GetSyncBootState();
}
}

19
src/Umbraco.Core/Sync/NonRuntimeLevelBootStateAccessor.cs Normal file
View File

@@ -0,0 +1,19 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Umbraco.Core.Sync
{
/// <summary>
/// Boot state implementation for when umbraco is not in the run state
/// </summary>
public class NonRuntimeLevelBootStateAccessor : ISyncBootStateAccessor
{
public SyncBootState GetSyncBootState()
{
return SyncBootState.Unknown;
}
}
}

24
src/Umbraco.Core/Sync/SyncBootState.cs Normal file
View File

@@ -0,0 +1,24 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Umbraco.Core.Sync
{
public enum SyncBootState
{
/// <summary>
/// Unknown state. Treat as HasSyncState
/// </summary>
Unknown = 0,
/// <summary>
/// Cold boot. No Sync state
/// </summary>
ColdBoot = 1,
/// <summary>
/// Warm boot. Sync state present
/// </summary>
HasSyncState = 2
}
}

3
src/Umbraco.Core/Umbraco.Core.csproj
View File

@@ -190,6 +190,9 @@
<Compile Include="Migrations\Upgrade\V_8_7_0\MissingDictionaryIndex.cs" />
<Compile Include="Services\IInstallationService.cs" />
<Compile Include="Services\IUpgradeService.cs" />
<Compile Include="Sync\ISyncBootStateAccessor.cs" />
<Compile Include="Sync\NonRuntimeLevelBootStateAccessor.cs" />
<Compile Include="Sync\SyncBootState.cs" />
<Compile Include="SystemLock.cs" />
<Compile Include="Attempt.cs" />
<Compile Include="AttemptOfTResult.cs" />

2
src/Umbraco.Tests/PublishedContent/NuCacheChildrenTests.cs
View File

@@ -17,6 +17,7 @@ using Umbraco.Core.Scoping;
using Umbraco.Core.Services;
using Umbraco.Core.Services.Changes;
using Umbraco.Core.Strings;
using Umbraco.Core.Sync;
using Umbraco.Tests.TestHelpers;
using Umbraco.Tests.Testing.Objects;
using Umbraco.Tests.Testing.Objects.Accessors;
@@ -158,6 +159,7 @@ namespace Umbraco.Tests.PublishedContent
Mock.Of<IEntityXmlSerializer>(),
Mock.Of<IPublishedModelFactory>(),
new UrlSegmentProviderCollection(new[] { new DefaultUrlSegmentProvider() }),
new TestSyncBootStateAccessor(SyncBootState.HasSyncState),
_contentNestedDataSerializerFactory);
// invariant is the current default

2
src/Umbraco.Tests/PublishedContent/NuCacheTests.cs
View File

@@ -17,6 +17,7 @@ using Umbraco.Core.Scoping;
using Umbraco.Core.Services;
using Umbraco.Core.Services.Changes;
using Umbraco.Core.Strings;
using Umbraco.Core.Sync;
using Umbraco.Tests.TestHelpers;
using Umbraco.Tests.Testing.Objects;
using Umbraco.Tests.Testing.Objects.Accessors;
@@ -204,6 +205,7 @@ namespace Umbraco.Tests.PublishedContent
Mock.Of<IEntityXmlSerializer>(),
Mock.Of<IPublishedModelFactory>(),
new UrlSegmentProviderCollection(new[] { new DefaultUrlSegmentProvider() }),
new TestSyncBootStateAccessor(SyncBootState.HasSyncState),
_contentNestedDataSerializerFactory);
// invariant is the current default

1
src/Umbraco.Tests/Scoping/ScopedNuCacheTests.cs
View File

@@ -101,6 +101,7 @@ namespace Umbraco.Tests.Scoping
Factory.GetInstance<IEntityXmlSerializer>(),
Mock.Of<IPublishedModelFactory>(),
new UrlSegmentProviderCollection(new[] { new DefaultUrlSegmentProvider() }),
new TestSyncBootStateAccessor(SyncBootState.HasSyncState),
nestedContentDataSerializerFactory);
}

2
src/Umbraco.Tests/Services/ContentTypeServiceVariantsTests.cs
View File

@@ -17,6 +17,7 @@ using Umbraco.Core.PropertyEditors;
using Umbraco.Core.Services;
using Umbraco.Core.Strings;
using Umbraco.Core.Sync;
using Umbraco.Tests.TestHelpers;
using Umbraco.Tests.TestHelpers.Entities;
using Umbraco.Tests.Testing;
using Umbraco.Web.PublishedCache;
@@ -73,6 +74,7 @@ namespace Umbraco.Tests.Services
Factory.GetInstance<IEntityXmlSerializer>(),
Mock.Of<IPublishedModelFactory>(),
new UrlSegmentProviderCollection(new[] { new DefaultUrlSegmentProvider() }),
new TestSyncBootStateAccessor(SyncBootState.HasSyncState),
nestedContentDataSerializerFactory);
}

23
src/Umbraco.Tests/TestHelpers/TestSyncBootStateAccessor.cs Normal file
View File

@@ -0,0 +1,23 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Umbraco.Core.Sync;
namespace Umbraco.Tests.TestHelpers
{
class TestSyncBootStateAccessor : ISyncBootStateAccessor
{
private readonly SyncBootState _syncBootState;
public TestSyncBootStateAccessor(SyncBootState syncBootState)
{
_syncBootState = syncBootState;
}
public SyncBootState GetSyncBootState()
{
return _syncBootState;
}
}
}

1
src/Umbraco.Tests/Umbraco.Tests.csproj
View File

@@ -184,6 +184,7 @@
<Compile Include="Services\RedirectUrlServiceTests.cs" />
<Compile Include="Templates\HtmlLocalLinkParserTests.cs" />
<Compile Include="TestHelpers\RandomIdRamDirectory.cs" />
<Compile Include="TestHelpers\TestSyncBootStateAccessor.cs" />
<Compile Include="Testing\Objects\TestDataSource.cs" />
<Compile Include="Published\PublishedSnapshotTestObjects.cs" />
<Compile Include="Published\ModelTypeTests.cs" />

1
src/Umbraco.Web/Compose/DatabaseServerRegistrarAndMessengerComponent.cs
View File

@@ -72,6 +72,7 @@ namespace Umbraco.Web.Compose
composition.SetDatabaseServerMessengerOptions(GetDefaultOptions);
composition.SetServerMessenger<BatchedDatabaseServerMessenger>();
composition.Register<ISyncBootStateAccessor>(factory=> factory.GetInstance<IServerMessenger>() as BatchedDatabaseServerMessenger, Lifetime.Singleton);
}
}

8
src/Umbraco.Web/PublishedCache/NuCache/DataSource/BTree.DictionaryOfPropertyDataSerializer.cs
View File

@@ -13,11 +13,11 @@ namespace Umbraco.Web.PublishedCache.NuCache.DataSource
{
public IDictionary<string, PropertyData[]> ReadFrom(Stream stream)
{
var dict = new Dictionary<string, PropertyData[]>(StringComparer.InvariantCultureIgnoreCase);
// read properties count
var pcount = PrimitiveSerializer.Int32.ReadFrom(stream);
var dict = new Dictionary<string, PropertyData[]>(pcount,StringComparer.InvariantCultureIgnoreCase);
// read each property
for (var i = 0; i < pcount; i++)
{
@@ -28,13 +28,13 @@ namespace Umbraco.Web.PublishedCache.NuCache.DataSource
var vcount = PrimitiveSerializer.Int32.ReadFrom(stream);
// create pdata and add to the dictionary
var pdatas = new List<PropertyData>();
var pdatas = new PropertyData[vcount];
// for each value, read and add to pdata
for (var j = 0; j < vcount; j++)
{
var pdata = new PropertyData();
pdatas.Add(pdata);
pdatas[j] =pdata;
// everything that can be null is read/written as object
// even though - culture and segment should never be null here, as 'null' represents
@@ -46,7 +46,7 @@ namespace Umbraco.Web.PublishedCache.NuCache.DataSource
pdata.Value = ReadObject(stream);
}
dict[key] = pdatas.ToArray();
dict[key] = pdatas;
}
return dict;
}

53
src/Umbraco.Web/PublishedCache/NuCache/DataSource/JsonContentNestedDataSerializer.cs
View File

@@ -1,6 +1,8 @@
using Newtonsoft.Json;
using System;
using System.Buffers;
using System.Collections.Generic;
using System.IO;
using Umbraco.Core.Models;
using Umbraco.Core.Serialization;
@@ -21,13 +23,20 @@ namespace Umbraco.Web.PublishedCache.NuCache.DataSource
DateTimeZoneHandling = DateTimeZoneHandling.Utc,
DateFormatString = "o"
};
private readonly JsonNameTable _propertyNameTable = new DefaultJsonNameTable();
public ContentCacheDataModel Deserialize(IReadOnlyContentBase content, string stringData, byte[] byteData)
{
if (stringData == null && byteData != null)
throw new NotSupportedException($"{typeof(JsonContentNestedDataSerializer)} does not support byte[] serialization");
return JsonConvert.DeserializeObject<ContentCacheDataModel>(stringData, _jsonSerializerSettings);
JsonSerializer serializer = JsonSerializer.Create(_jsonSerializerSettings);
using (JsonTextReader reader = new JsonTextReader(new StringReader(stringData)))
{
// reader will get buffer from array pool
reader.ArrayPool = JsonArrayPool.Instance;
reader.PropertyNameTable = _propertyNameTable;
return serializer.Deserialize<ContentCacheDataModel>(reader);
}
}
public ContentCacheDataSerializationResult Serialize(IReadOnlyContentBase content, ContentCacheDataModel model)
@@ -39,4 +48,44 @@ namespace Umbraco.Web.PublishedCache.NuCache.DataSource
return new ContentCacheDataSerializationResult(json, null);
}
}
public class JsonArrayPool : IArrayPool<char>
{
public static readonly JsonArrayPool Instance = new JsonArrayPool();
public char[] Rent(int minimumLength)
{
// get char array from System.Buffers shared pool
return ArrayPool<char>.Shared.Rent(minimumLength);
}
public void Return(char[] array)
{
// return char array to System.Buffers shared pool
ArrayPool<char>.Shared.Return(array);
}
}
public class AutomaticJsonNameTable : DefaultJsonNameTable
{
int nAutoAdded = 0;
int maxToAutoAdd;
public AutomaticJsonNameTable(int maxToAdd)
{
this.maxToAutoAdd = maxToAdd;
}
public override string Get(char[] key, int start, int length)
{
var s = base.Get(key, start, length);
if (s == null && nAutoAdded < maxToAutoAdd)
{
s = new string(key, start, length);
Add(s);
nAutoAdded++;
}
return s;
}
}
}

4
src/Umbraco.Web/PublishedCache/NuCache/NuCacheComposer.cs
View File

@@ -1,6 +1,7 @@
using System.Configuration;
using Umbraco.Core;
using Umbraco.Core.Composing;
using Umbraco.Core.Sync;
using Umbraco.Core.PropertyEditors;
using Umbraco.Web.PublishedCache.NuCache.DataSource;
@@ -27,6 +28,9 @@ namespace Umbraco.Web.PublishedCache.NuCache
composition.RegisterUnique(factory => new ContentDataSerializer(new DictionaryOfPropertyDataSerializer()));
//Overriden on Run state in DatabaseServerRegistrarAndMessengerComposer
composition.Register<ISyncBootStateAccessor, NonRuntimeLevelBootStateAccessor>(Lifetime.Singleton);
// register the NuCache database data source
composition.RegisterUnique<IDataSource, DatabaseDataSource>();

19
src/Umbraco.Web/PublishedCache/NuCache/PublishedSnapshotService.cs Executable file → Normal file
View File

@@ -21,6 +21,7 @@ using Umbraco.Core.Services;
using Umbraco.Core.Services.Changes;
using Umbraco.Core.Services.Implement;
using Umbraco.Core.Strings;
using Umbraco.Core.Sync;
using Umbraco.Web.Cache;
using Umbraco.Web.Install;
using Umbraco.Web.PublishedCache.NuCache.DataSource;
@@ -62,6 +63,8 @@ namespace Umbraco.Web.PublishedCache.NuCache
private bool _localContentDbExists;
private bool _localMediaDbExists;
private readonly ISyncBootStateAccessor _syncBootStateAccessor;
// define constant - determines whether to use cache when previewing
// to store eg routes, property converted values, anything - caching
// means faster execution, but uses memory - not sure if we want it
@@ -80,7 +83,10 @@ namespace Umbraco.Web.PublishedCache.NuCache
IDataSource dataSource, IGlobalSettings globalSettings,
IEntityXmlSerializer entitySerializer,
IPublishedModelFactory publishedModelFactory,
UrlSegmentProviderCollection urlSegmentProviders, IContentCacheDataSerializerFactory contentCacheDataSerializerFactory, ContentDataSerializer contentDataSerializer = null)
UrlSegmentProviderCollection urlSegmentProviders,
ISyncBootStateAccessor syncBootStateAccessor,
IContentCacheDataSerializerFactory contentCacheDataSerializerFactory,
ContentDataSerializer contentDataSerializer = null)
: base(publishedSnapshotAccessor, variationContextAccessor)
{
//if (Interlocked.Increment(ref _singletonCheck) > 1)
@@ -100,6 +106,8 @@ namespace Umbraco.Web.PublishedCache.NuCache
_contentCacheDataSerializerFactory = contentCacheDataSerializerFactory;
_contentDataSerializer = contentDataSerializer;
_syncBootStateAccessor = syncBootStateAccessor;
// we need an Xml serializer here so that the member cache can support XPath,
// for members this is done by navigating the serialized-to-xml member
_entitySerializer = entitySerializer;
@@ -218,7 +226,12 @@ namespace Umbraco.Web.PublishedCache.NuCache
{
var okContent = false;
var okMedia = false;
if (_syncBootStateAccessor.GetSyncBootState() == SyncBootState.ColdBoot)
{
_logger.Warn<PublishedSnapshotService>("Sync Service is in a Cold Boot state. Skip LoadCachesOnStartup as the Sync Service will trigger a full reload");
_isReady = true;
return;
}
try
{
if (_localContentDbExists)
@@ -234,7 +247,7 @@ namespace Umbraco.Web.PublishedCache.NuCache
if (!okMedia)
_logger.Warn<PublishedSnapshotService>("Loading media from local db raised warnings, will reload from database.");
}
if (!okContent)
LockAndLoadContent(scope => LoadContentFromDatabaseLocked(scope, true));