Using DataManager

DataManager is the main interface for CRUD (Create, Read, Update, Delete) operations on entities. It allows you to load graphs of entities by ID or query, to save changed instances or remove them. You can use entity event listeners to perform actions on loading and saving of particular entities. DataManager maintains cross-datastore references for both JPA, DTO, and mixed entity graphs.

You can inject DataManager to a Spring bean or a screen controller, for example:

@Component
public class CustomerService {

    @Autowired
    private DataManager dataManager;

In the examples below, we skip the definition and assume that the dataManager variable is a reference to DataManager.

Loading Entities

DataManager provides a fluent API for loading entities. Use load() methods accepting entity class or Id as entry points to this API.

Loading Entity by Id

The following method loads an entity by its identifier value:

Customer loadById(UUID customerId) {
    return dataManager.load(Customer.class) (1)
            .id(customerId)                 (2)
            .one();                         (3)
}
1 Entry point to the fluent loader API.
2 id() method accepts the identifier value.
3 one() method loads the entity instance. If there is no entity with the given identifier, the method throws IllegalStateException.

The identifier can also be specified using Id<E> class which contains information about the entity type. Then the application code doesn’t need to use concrete type of the entity identifier (UUID, Long, etc.) and the loading code becomes even shorter:

Customer loadByGenericId(Id<Customer> customerId) {
    return dataManager.load(customerId).one();
}

If the entity with the given identifier may not exist, instead of one() terminal method use optional() which returns Optional<E>. In the example below, if the entity not found, a new instance is created and returned:

Customer loadOrCreate(UUID customerId) {
    return dataManager.load(Customer.class)
            .id(customerId)
            .optional() (1)
            .orElse(dataManager.create(Customer.class));
}
1 Returns Optional<Customer>.

You can also load a list of entities by their identifiers passed to the ids() method, for example:

List<Customer> loadByIds(UUID id1, UUID id2) {
    return dataManager.load(Customer.class)
            .ids(id1, id2)
            .list();
}

Entities in the result list have the same order as provided identifiers.

Loading All Entities

The following method loads all instances of the entity in a list:

List<Customer> loadAll() {
    return dataManager.load(Customer.class).all().list();
}
Load all instances only if you are sure that the number of rows in the corresponding table is always low. Otherwise, use a query, conditions and/or paging.

Loading Entities by Query

When working with relational databases, use JPQL queries to load data. See the JPQL Extensions for information on how JPQL in Jmix differs from the JPA standard. Also note that DataManager can execute only "select" queries.

The following method loads a list of entities using a full JPQL query and two parameters:

List<Customer> loadByFullQuery() {
    return dataManager.load(Customer.class)
            .query("select c from sample_Customer c where c.email like :email and c.grade = :grade")
            .parameter("email", "%@company.com")
            .parameter("grade", CustomerGrade.PLATINUM)
            .list();
}

The query() method of the fluent loader interface accepts both full and abbreviated query string. The latter should be created in accordance with the following rules:

  • You can always omit the "select <alias>" clause.

  • If the "from" clause contains a single entity and you don’t need a specific alias for it, you can omit the "from <entity> <alias> where" clause. In this case, the framework assumes that the alias is e.

  • You can use positional parameters and pass their values right to the query() method as additional arguments.

Below is an abbreviated equivalent of the previous example:

List<Customer> loadByQuery() {
    return dataManager.load(Customer.class)
            .query("e.email like ?1 and e.grade = ?2", "%@company.com", CustomerGrade.PLATINUM)
            .list();
}

An example of a more complex abbreviated query with join:

List<Order> loadOrdersByProduct(String productName) {
    return dataManager.load(Order.class)
            .query("from sample_Order o, sample_OrderLine l " +
                    "where l.order = o and l.product.name = ?1", productName)
            .list();
}

Loading Entities by Conditions

You can use conditions instead of a JPQL query to filter results. For example:

List<Customer> loadByConditions() {
    return dataManager.load(Customer.class)
            .condition(                                                      (1)
                LogicalCondition.and(                                        (2)
                    PropertyCondition.contains("email", "@company.com"),     (3)
                    PropertyCondition.equal("grade", CustomerGrade.PLATINUM) (3)
                )
            )
            .list();
}
1 condition() method accepts a single root condition.
2 LogicalCondition.and() method creates AND condition with the given nested conditions.
3 Property conditions compare entity attributes with the specified values.

If you need a single property condition, pass it directly to the condition() method:

List<Customer> loadByCondition() {
    return dataManager.load(Customer.class)
            .condition(PropertyCondition.contains("email", "@company.com"))
            .list();
}

PropertyCondition lets you specify properties of referenced entities, for example:

List<Order> loadByCondition() {
    return dataManager.load(Order.class)
            .condition(PropertyCondition.contains("customer.email", "@company.com"))
            .list();
}

Loading Scalar and Aggregate Values

Besides entity instances, DataManager can load scalar and aggregate values in the form of key-value entities.

The loadValues(String query) method loads a list of KeyValueEntity instances populated with a given query results. For example:

String getCustomerPurchases(LocalDate fromDate) {
    List<KeyValueEntity> kvEntities = dataManager.loadValues(
            "select o.customer, sum(o.amount) from sample_Order o " +
                    "where o.date >= :date group by o.customer")
            .store("main")                      (1)
            .properties("customer", "sum")      (2)
            .parameter("date", fromDate)
            .list();

    StringBuilder sb = new StringBuilder();
    for (KeyValueEntity kvEntity : kvEntities) {
        Customer customer = kvEntity.getValue("customer");  (3)
        BigDecimal sum = kvEntity.getValue("sum");          (3)
        sb.append(customer.getName()).append(" : ").append(sum).append("\n");
    }
    return sb.toString();
}
1 Specify data store where the requested entities are located. Omit this method if the entity is located in the main data store.
2 Specify names of the resulting key-value entity attributes. The order of the properties must correspond to the columns in the query result set.
3 Get loaded values from the key-value entity attributes.

The loadValue(String query, Class valueType) method loads a single value of the given type by the query. For example:

BigDecimal getTotal(LocalDate toDate) {
    return dataManager.loadValue(
                "select sum(o.amount) from sample_Order o where o.date >= :date",
                BigDecimal.class    (1)
            )
            .store("main")          (2)
            .parameter("date", toDate)
            .one();
}
1 The type of the returned value.
2 Specify data store where the requested entities are located. Omit this method if the entity is located in the main data store.

Limitations

The loadValues() and loadValue() have the following limitations:

  1. The query must be a valid JPQL query for the single specified data store. It can contain only persistent attributes of JPA entities.

  2. Cross-datastore references are not supported.

  3. These methods are not implemented in REST DataStore.

Paging and Sorting

When you load entities using all(), query() or condition() methods, you can also sort results and split them into pages.

Use firstResult() and maxResults() methods for paging:

List<Customer> loadPageByQuery(int offset, int limit) {
    return dataManager.load(Customer.class)
            .query("e.grade = ?1", CustomerGrade.BRONZE)
            .firstResult(offset)
            .maxResults(limit)
            .list();
}

Use sort() method for sorting results:

List<Customer> loadSorted() {
    return dataManager.load(Customer.class)
            .condition(PropertyCondition.contains("email", "@company.com"))
            .sort(Sort.by("name"))
            .list();
}

In the Sort.by() method, you can specify properties of referenced entities, for example:

List<Order> loadSorted() {
    return dataManager.load(Order.class)
            .all()
            .sort(Sort.by("customer.name"))
            .list();
}

When loading by a JPQL query, you can also use the standard order by clause in the query:

List<Customer> loadByQuerySorted() {
    return dataManager.load(Customer.class)
            .query("e.grade = ?1 order by e.name", CustomerGrade.BRONZE)
            .list();
}

Using Locks

The lockMode() method accepting the jakarta.persistence.LockModeType enum values can be used for locking JPA entities on the database level. For example, the following example obtains a pessimistic lock using a select …​ for update SQL statement:

List<Customer> loadAndLock() {
    return dataManager.load(Customer.class)
            .query("e.email like ?1", "%@company.com")
            .lockMode(LockModeType.PESSIMISTIC_WRITE)
            .list();
}

Saving Entities

Use save() method to save new and modified entities in the database.

In the simplest form, the method accepts an entity instance and returns a saved instance:

Customer saveCustomer(Customer entity) {
    return dataManager.save(entity);
}
Usually the passed and returned instances are not the same. The returned instance can be affected by entity event listeners, database triggers or access control rights. So if you need to save an entity and then continue working with it, always use the instance returned from the save() method.

The save() method can accept multiple instances at once. In this case, it returns the EntitySet object which can be used to easily get the saved instances. In the example below, we create and save two linked entities, and return one of them:

Order createOrderWithCustomer() {
    Customer customer = dataManager.create(Customer.class);
    customer.setName("Alice");

    Order order = dataManager.create(Order.class);
    order.setCustomer(customer);

    EntitySet savedEntities = dataManager.save(order, customer); (1)

    return savedEntities.get(order); (2)
}
1 Save two linked entities. The order of save() parameters is not important.
2 The EntitySet.get() method lets you obtain the saved instance by its source instance.

The most powerful form of the save() method accepts SaveContext object that can be used to add multiple instances and specify additional parameters of saving. In the example below, we save a collection of entities using SaveContext:

EntitySet saveUsingContext(List<Customer> entities) {
    SaveContext saveContext = new SaveContext();
    for (Customer entity : entities) {
        saveContext.saving(entity);
    }
    return dataManager.save(saveContext);
}

Save Performance

There are a few techniques for improving performance of the save operation. They are especially useful if you are working with a large collection of entities.

First of all, instead of passing each entity instance to separate calls of the save(entity) method, consider saving all entities (if the collection is not very large, say, less than 1000) in a single transaction: add entities to the SaveContext and use the save(SaveContext) method as explained in the previous section.

If you don’t need saved instances immediately, use SaveContext.setDiscardSaved(true) method. It will significantly improve performance because DataManager will not fetch saved entities back from the database. For example:

void saveAndReturnNothing(List<Customer> entities) {
    // create SaveContext and set its 'discardSaved' property
    SaveContext saveContext = new SaveContext().setDiscardSaved(true);
    for (Customer entity : entities) {
        saveContext.saving(entity);
    }
    dataManager.save(saveContext);
}

If you don’t need to check current user security permissions, use UnconstrainedDataManager to get an additional performance gain. For example:

void saveByUnconstrainedDataManager(List<Customer> entities) {
    SaveContext saveContext = new SaveContext().setDiscardSaved(true);
    for (Customer entity : entities) {
        saveContext.saving(entity);
    }
    // use 'UnconstrainedDataManager' which bypasses security
    dataManager.unconstrained().save(saveContext);
}

If the collection is large (more than 1000 instances), it is vital for the good performance to split the save operation to batches. For example:

void saveInBatches(List<Customer> entities) {
    SaveContext saveContext = new SaveContext().setDiscardSaved(true);
    for (int i = 0; i < entities.size(); i++) {
        saveContext.saving(entities.get(i));
        // save by 100 instances
        if ((i + 1) % 100 == 0 || i == entities.size() - 1) {
            dataManager.save(saveContext);
            saveContext = new SaveContext().setDiscardSaved(true);
        }
    }
}

See the comparison of different saving methods in the jmix-data-performance-tests project on GitHub.

Removing Entities

Use remove() method to remove entities from the database.

In the simplest form, the method accepts an entity instance to remove:

void removeCustomer(Customer entity) {
    dataManager.remove(entity);
}

The remove() method can accept multiple instances, arrays and collections:

void removeCustomers(List<Customer> entities) {
    dataManager.remove(entities);
}

If you remove linked entities, the order of parameters can be important. First pass the entities that depend on the others, for example:

void removeOrderWithCustomer(Order order) {
    dataManager.remove(order, order.getCustomer());
}

If you don’t have an entity instance but only its identifier, construct the Id object from the identifier and pass it to the remove() method:

void removeCustomer(UUID customerId) {
    dataManager.remove(Id.of(customerId, Customer.class));
}

If you need to specify additional parameters of the remove operation, for example to turn off soft deletion and completely remove from the database an entity with the soft delete trait, use the save() method with SaveContext and pass the removed entities to its removing() method:

void hardDelete(Product product) {
    dataManager.save(
            new SaveContext()
                    .removing(product)
                    .setHint(PersistenceHints.SOFT_DELETION, false)
    );
}

Transactions in DataManager

When working with JPA entities, DataManager by default joins the current transaction or creates and commits a new one if there is no existing transaction.

You can manage transactions using annotations or TransactionTemplate as described in the Transaction Management section.

Besides, DataManager allows you to control its own internal transactional behavior.

When loading data using the fluent API, you can use the joinTransaction(false) method to begin and commit a separate transaction for this operation:

Customer loadCustomerInSeparateTransaction(UUID customerId) {
    return dataManager.load(Customer.class)
            .id(customerId)
            .joinTransaction(false)
            .one();
}

When saving entities, use the setJoinTransaction(false) method of SaveContext to to begin and commit a separate transaction for the operation with this context:

void saveCustomerInSeparateTransaction(Customer entity) {
    SaveContext saveContext = new SaveContext().saving(entity)
            .setJoinTransaction(false);
    dataManager.save(saveContext);
}

Security in DataManager

DataManager performs the entity policy checks:

  • If the user has no rights to CREATE, UPDATE or DELETE action, the save() and remove() methods will throw the io.jmix.core.security.AccessDeniedException exception on the corresponding operation.

  • If the user has no rights to the READ action, the load() methods will return the empty result: null or an empty list. This applies only to the root entity of the loaded object graph; the references are always loaded.

DataManager also respects the row-level policies. The JPQL policy affects only the root entity of the loaded object graph, while the predicate policy affects the root entity and all linked entities.

The UnconstrainedDataManager interface has the same methods as DataManager but it doesn’t check the security policies. You can use it instead of the DataManager to bypass the security in your code, for example:

@Autowired
private UnconstrainedDataManager unconstrainedDataManager;

public Customer loadByIdUnconstrained(UUID customerId) {
    return unconstrainedDataManager.load(Customer.class)
            .id(customerId)
            .one();
}

See more information about security in the Data Access Checks section.