Data Sources

In Coalesce, all data that is retrieved from your database through the generated controllers is done so by a data source. These data sources control what data gets loaded and how it gets loaded. By default, there is a single generic data source that serves all data for your models in a generic way that fits many of the most common use cases - the Standard Data Source.

In addition to this standard data source, Coalesce allows you to create custom data sources that provide complete control over the way data is loaded and serialized for transfer to a requesting client. These data sources are defined on a per-model basis, and you can have as many of them as you like for each model.

Defining Data Sources

By default, each of your models that Coalesce exposes will expose the standard data source (IntelliTect.Coalesce.StandardDataSource<T, TContext>). This data source provides all the standard functionality one would expect - paging, sorting, searching, filtering, and so on. Each of these component pieces is implemented in one or more virtual methods, making the StandardDataSource a great place to start from when implementing your own data source. To suppress this behavior of always exposing the raw StandardDataSource, create your own custom data source and annotate it with [DefaultDataSource].

To implement your own custom data source, you simply need to define a class that implements IntelliTect.Coalesce.IDataSource<T>. To expose your data source to Coalesce, either place it as a nested class of the type T that you data source serves, or annotate it with the [Coalesce] attribute. Of course, the easiest way to create a data source that doesn’t require you to re-engineer a great deal of logic would be to inherit from IntelliTect.Coalesce.StandardDataSource<T, TContext>, and then override only the parts that you need.

public class Person
{
    [DefaultDataSource]
    public class IncludeFamily : StandardDataSource<Person, AppDbContext>
    {
        public IncludeFamily(CrudContext<AppDbContext> context) : base(context) { }

        public override IQueryable<Person> GetQuery(IDataSourceParameters parameters)
            => Db.People
            .Where(f => User.IsInRole("Admin") || f.CreatedById == User.GetUserId())
            .Include(f => f.Parents).ThenInclude(s => s.Parents)
            .Include(f => f.Cousins).ThenInclude(s => s.Parents);
    }
}

[Coalesce]
public class NamesStartingWithA : StandardDataSource<Person, AppDbContext>
{
    public NamesStartingWithA(CrudContext<AppDbContext> context) : base(context) { }

    public override IQueryable<Person> GetQuery(IDataSourceParameters parameters)
        => Db.People.Include(f => f.Siblings).Where(f => f.FirstName.StartsWith("A"));
}

The structure of the IQueryable built by the various methods of StandardDataSource is used to shape and trim the structure of the DTO as it is serialized and sent out to the client. One may also override method IncludeTree GetIncludeTree(IQueryable<Person> query, IDataSourceParameters parameters) to control this explicitly. See Include Tree for more information on how this works.

Warning

If you create a custom data source that has custom logic for securing your data, be aware that the default implementation of StandardDataSource (or your custom default implementation - see below) is still exposed unless you annotate one of your custom data sources with [DefaultDataSource]. Doing so will replace the default data source with the annotated class for your type T.

Dependency Injection

All data sources are instantiated using dependency injection and your application’s IServiceProvider. As a result, you can add whatever constructor parameters you desire to your data sources as long as a value for them can be resolved from your application’s services. The single parameter to the StandardDataSource is resolved in this way - the CrudContext<TContext> contains the common set of objects most commonly used, including the DbContext and the ClaimsPrincipal representing the current user.

Consuming Data Sources

The TypeScript ViewModels and ListViewModels have a property called dataSource. These properties accept an instance of a Coalesce.DataSource<T>. Generated classes that satisfy this relationship for all the data sources that were defined in C# may be found in the dataSources property on an instance of a ViewModel or ListViewModel, or in ListViewModels.<ModelName>DataSources

var viewModel = new ViewModels.Person();
viewModel.dataSource = new viewModel.dataSources.IncludeFamily();
viewModel.load(1);

var list = new ListViewModels.PersonList();
list.dataSource = new list.dataSources.NamesStartingWith();
list.load();

Standard Parameters

All methods on IDataSource<T> take a parameter that contains all the client-specified parameters for things paging, searching, sorting, and filtering information. Almost all overridable methods on StandardDataSource are also passed the relevant set of parameters.

Custom Parameters

On any data source that you create, you may add additional properties annotated with [Coalesce] that will then be exposed as parameters to the client. These property parameters are currently restricted to primitives (numeric types, strings) and dates (DateTime, DateTimeOffset). Property parameter primitives may be expanded to allow for more types in the future.

[Coalesce]
public class NamesStartingWith : StandardDataSource<Person, AppDbContext>
{
    public NamesStartingWith(CrudContext<AppDbContext> context) : base(context) { }

    [Coalesce]
    public string StartsWith { get; set; }

    public override IQueryable<Person> GetQuery(IDataSourceParameters parameters)
        => Db.People.Include(f => f.Siblings)
        .Where(f => string.IsNullOrWhitespace(StartsWith) ? true : f.FirstName.StartsWith(StartsWith));
}

The properties created on the TypeScript objects are observables so they may be bound to directly. In order to automatically reload a list when a data source parameter changes, you must explicitly subscribe to it:

var list = new ListViewModels.PersonList();
var dataSource = new list.dataSources.NamesStartingWith();
dataSource.startsWith("Jo");
dataSource.subscribe(list); // Optional - call to enable automatic reloading.
list.dataSource = dataSource;
list.load();

Standard Data Source

The standard data source, IntelliTect.Coalesce.StandardDataSource<T, TContext>, contains a significant number of properties and methods that can be utilized and/or overridden at your leisure.

Default Loading Behavior

When an object or list of objects is requested, the default behavior of the the StandardDataSource is to load all of the immediate relationships of the object (parent objects and child collections), as well as the far side of many-to-many relationships. This can be suppressed by settings includes = “none” on your TypeScript ViewModel or ListViewModel when making a request.

In most cases, however, you’ll probably want more or less data than what the default behavior provides. You can achieve this by overriding the GetQuery method, outlined below.

Properties

The following properties are available for use on the StandardDataSource

CrudContext<TContext> Context
The object passed to the constructor that contains the set of objects needed by the standard data source, and those that are most likely to be used in custom implementations.
TContext Db
An instance of the db context that contains a DbSet<T> for the entity served by the data source.
ClaimsPrincipal User
The user making the current request.
int MaxSearchTerms
The max number of search terms to process when interpreting a search term word-by-word. Override by setting a value in the constructor.
int DefaultPageSize
The page size to use if none is specified by the client. Override by setting a value in the constructor.
int MaxPageSize
The maximum page size that will be served. By default, client-specified page sizes will be clamped to this value. Override by setting a value in the constructor.

Method Overview

The standard data source contains 19 different methods which can be overridden in your derived class to control its behavior.

These methods often call one another, so overriding one method may cause some other method to no longer be called. The hierarchy of method calls, ignoring any logic or conditions contained within, is as follows:

GetMappedItemAsync
    GetItemAsync
        GetQuery
        GetIncludeTree
    TransformResults

GetMappedListAsync
    GetListAsync
        GetQuery
        ApplyListFiltering
            ApplyListPropertyFilters
                ApplyListPropertyFilter
            ApplyListSearchTerm
        GetListTotalCountAsync
        ApplyListSorting
            ApplyListClientSpecifiedSorting
            ApplyListDefaultSorting
        ApplyListPaging
        GetIncludeTree
    TrimListFields
    TransformResults

GetCountAsync
    GetQuery
    ApplyListFiltering
        ApplyListPropertyFilters
            ApplyListPropertyFilter
        ApplyListSearchTerm
    GetListTotalCountAsync

Method Details

All of the methods outlined above can be overridden. A description of each of the non-interface inner methods is as follows:

GetQuery

The method is the one that you will most commonly be override in order to implement custom query logic. From this method, you could:

  • Specify additional query filtering such as row-level security or soft-delete logic. Or, restrict the data source entirely for users or whole roles by returning an empty query.
  • Include additional data using EF’s .Include() and .ThenInclude().
  • Add additional edges to the serialized object graph using Coalesce’s .IncludedSeparately() and .ThenIncluded().

Note

When GetQuery is overridden, the Default Loading Behavior is overridden as well. To restore this behavior, use the IQueryable<T>.IncludeChildren() extension method to build your query.

GetIncludeTree
Allows for explicitly specifying the Include Tree that will be used when serializing results obtained from this data source into DTOs. By default, the query that is build up through all the other methods in the data source will be used to build the include tree.
CanEvalQueryAsynchronously
Called by other methods in the standard data source to determine whether or not EF Core async methods will be used to evaluate queries. This may be globally disabled when bugs like https://github.com/aspnet/EntityFrameworkCore/issues/9038 are present in EF Core.
ApplyListFiltering
A simple wrapper that calls ApplyListPropertyFilters and ApplyListSearchTerm.
ApplyListPropertyFilters
For each value in parameters.Filter, invoke ApplyListPropertyFilter to apply a filter to the query.
ApplyListPropertyFilter

Given a property and a client-provided string value, perform some filtering on that property.

  • Dates with a time component will be matched exactly.
  • Dates with no time component will match any dates that fell on that day.
  • Strings will match exactly unless an asterisk is found, in which case they will be matched with string.StartsWith.
  • Enums will match by string or numeric value. Multiple comma-delimited values will create a filter that will match on any of the provided values.
  • Numeric values will match exactly. Multiple comma-delimited values will create a filter that will match on any of the provided values.
ApplyListSearchTerm
Applies filters to the query based on the specified search term. See [Search] for a detailed look at how searching works in Coalesce.
ApplyListSorting
If any client-specified sort orders are present, invokes ApplyListClientSpecifiedSorting. Otherwise, invokes ApplyListDefaultSorting.
ApplyListClientSpecifiedSorting
Applies sorting to the query based on sort orders specified by the client. If the client specified "none" as the sort field, no sorting will take place.
ApplyListDefaultSorting
Applies default sorting behavior to the query, including behavior defined with use of [DefaultOrderBy] in C# POCOs, as well as fallback sorting to "Name" or primary key properties.
ApplyListPaging
Applies paging to the query based on incoming parameters. Provides the actual page and pageSize that were used as out parameters.
GetListTotalCountAsync
Simple wrapper around invoking .Count() on a query.
TransformResults

Allows for transformation of a result set after the query has been evaluated. This will be called for both lists of items and for single items. This can be used for things like populating non-mapped properties on a model. This method is only called immediately before mapping to a DTO - if the data source is serving data without mapping (e.g. when invoked by Behaviors) to a DTO, this will not be called..

Warning

It is STRONGLY RECOMMENDED that this method does not modify any database-mapped properties, as any such changes could be inadvertently persisted to the database.

TrimListFields
Performs trimming of the fields of the result set based on the parameters given to the data source. Can be overridden to forcibly disable this, override the behavior to always trim specific fields, or any other functionality desired.

Replacing the Standard Data Source

You can, of course, create a custom base data source that all your custom implementations inherit from. But, what if you want to override the standard data source across your entire application, so that StandardDataSource<,> will never be instantiated? You can do that too!

Simply create a class that implements IEntityFrameworkDataSource<,> (the StandardDataSource<,> already does - feel free to inherit from it), then register it at application startup like so:

public class MyDataSource<T, TContext> : StandardDataSource<T, TContext>
    where T : class, new()
    where TContext : DbContext
{
    public MyDataSource(CrudContext<TContext> context) : base(context)
    {
    }

    ...
}
public void ConfigureServices(IServiceCollection services)
{
    services.AddCoalesce(b =>
    {
        b.AddContext<AppDbContext>();
        b.UseDefaultDataSource(typeof(MyDataSource<,>));
    });

Your custom data source must have the same generic type parameters - <T, TContext>. Otherwise, the Microsoft.Extensions.DependencyInjection service provider won’t know how to inject it.