[[cil7]]
= Component Implementation - Lesson 7 - Data Repository
:author: not known
:version: v2.0, July 2015: Reformatted for v8.0.0.

== ConfigRepository
There are cases when you want to store some data permanently by your component. You can of course use the component configuration to provide some database connection settings, implement your own database connector and store records you need. There is, however, a very simple and useful framework which allows you to read and store some data transparently in either a database or a disk file. The framework also supports ad-hoc command interface straight away so you can manipulate your component data using an XMPP client.

In order to use it one needs to extend `tigase.db.comp.ConfigRepository` abstract class.

== Accessing UserRepository or AuthRepository
To use *AuthRepository* or *UserRepository* you need only to declare fields properly and annotated them with *@Inject*. This fields must be part of a class managed by Tigase Kernel - class of a component or any class annotated with *@Bean* annotation.
For that classes proper instances of repositories will be injected by dependency injection.

.Example usage of AuthRepository and UserRepository
[source,java]
----
@Inject
private AuthRepository authRepository;
@Inject
private UserRepository userRepository;
----

== Accessing other repositories
In order to have more freedom while accessing repositories it's possible to create and use custom repository implementation which implements *DataSourceAware* interface.

For our example let's assume it will be class implementing *TestRepositoryIfc* and our implementation will be using JDBC.
To make it work, we need to define `TestRepositoryIfc` as a generic interface extending `DataSourceAware` interface.
`DataSourceAware` interface will provide definition for methods required by Tigase XMPP Server internals to initialize custom repository classes based on `TestRepositoryIfc`.

.TestRepositoryIfc
[source,java]
----
public interface TestRepositoryIfc<DS extends DataSource> extends DataSourceAware<DS> {
  // Example method
  void addItem(BareJID userJid, String item) throws RepositoryException;
}
----

Next we need to prepare our actual implementation of repository - class responsible for execution of SQL statements. In this class we need to implement all of methods from our interface
and method *void setDataSource(DataSource dataSource)* which comes from *DataSourceAware* interface. In this method we need to initialize data source, ie. create prepared statements.
We should annotate our new class with `@Repository.Meta` annotation which will allow Tigase XMPP Server to find this class whenever class implementing `TestRepositoryIfc` and with support for data source with jdbc URI.

[source,java]
----
@Repository.Meta(supportedUris = "jdbc:.*")
public static class JDBCTestRepository implements TestRepositoryIfc<DataRepository> {

  private static final String SOME_STATEMENT = "select * from tig_users";

  private DataRepository repository;

  @Override
  public void setDataSource(DataRepository repository) {
    // here we need to initialize required prepared statements
    try {
      repository.initPreparedStatement(SOME_STATEMENT, SOME_STATEMENT);
    } catch (SQLException ex) {
      throw new RuntimeException("Could not initialize repository", ex);
    }
    this.repository = repository;
  }

  @Override
  public void addItem(BareJID userJid, String item) throws RepositoryException {
    try {
      PreparedStatement stmt = repository.getPreparedStatement(userJid, SOME_STATEMENT);
      synchronized (stmt) {
        // do what needs to be done
      }
    } catch (SQLException ex) {
      throw new RepositoryException(ex);
    }
  }
}
----

As you can see we defined type of a data source generic parameter for interface `TestRepositoryIfc`. With that we make sure that only instance implementing `DataRepository` interface will be provided and thanks to that we do not need to cast provided instance of `DataSource` to this interface before any access to data source.

With that in place we need to create class which will take care of adding support for multi-database setup. In our case it will be *TestRepositoryMDBean*, which will take care of discovery of repository class, initialization and re-injection of data source.
It is required to do so, as it was just mentioned our `TestRepositoryMDBean` will be responsible for initialization of `JDBCTestRepository` (actually this will be done by `MDRepositoryBean` which is extended by `TestRepositoryMDBean`.

[source,java]
----
@Bean(name = "repository", parent = TestComponent.class, active = true)
public static class TestRepositoryMDBean extends MDRepositoryBeanWithStatistics<TestRepositoryIfc>
    implements TestRepositoryIfc {

  public TestRepositoryMDBean() {
    super(TestRepositoryIfc.class);
  }

  @Override
  public Class<?> getDefaultBeanClass() {
    return TestRepositoryConfigBean.class;
  }

  @Override
  public void setDataSource(DataSource dataSource) {
    // nothing to do here
  }

  @Override
  public void addItem(BareJID userJid, String item) throws RepositoryException {
    getRepository(userJid.getDomain()).addItem(userJid, item);
  }

  @Override
  protected Class<? extends TestRepositoryIfc> findClassForDataSource(DataSource dataSource)
				throws DBInitException {
    return DataSourceHelper.getDefaultClass(TestRepositoryIfc.class, dataSource.getResourceUri());
  }

  public static class TestRepositoryConfigBean extends MDRepositoryConfigBean<TestRepositoryIfc> {
  }
}
----

Most of this code will be the same in all implementations based on `MDRepositoryBeanWithStatistics`. In our case only custom method is *void addItem(...)* which uses *getRepository(String domain)* method to retrieve correct repository for a domain.
This retrieval of actual repository instance for a domain will need to be done for every custom method of `TestRepositoryIfc`.

TIP: It is also possible to extend `MDRepositoryBean` or `SDRepositoryBean` instead of `MDRepositoryBeanWithStatistics`. However, if you decide to extend abstract repository bean classes without `withStatistics` suffix, then no statistics data related to usage of this repository will be gathered. The only change, will be that you will not need to pass interface class to constructor of a superclass as it is not needed.

NOTE: As mentioned above, it is also possible to extend `SDRepostioryBean` and `SDRepositoryBeanWithStatistics`. Methods which you would need to implement are the same is in case of extending `MDRepositoryBeanWithStatistics`, however internally `SDRepositoryBean` will not have support for using different repository for different domain. In fact `SDRepositoryBeanWithStatistics` has only one repository instance and uses only one data source for all domains.
The same behavior is presented by `MDRepositoryBeanWithStatistics` if only single `default` instance of repository is configured. However, `MDRepositoryBeanWithStatistics` gives better flexibility and due to that usage of `SDRepositoryBean` and `SDRepositoryBeanWithStatistics` is discouraged.

While this is more difficult to implement than in previous version, it gives you support for multi database setup and provides you with statistics of database query times which may be used for diagnosis.

As you can also see, we've annotated *TestRepositoryMDBean* with *@Bean* annotation which will force Tigase Kernel to load it every time *TestComponent* will be loaded. This way it is possible to inject instance of this class as a dependency to any bean used by this component (ie. component, module, etc.) by just creating a field and annotating it:

[source,java]
----
@Inject
private TestRepositoryIfc testRepository;
----

TIP: In *testRepository* field instance of *TestRepositoryMDBean* will be injected.

NOTE: If the class in which we intend to use our repository is deeply nested within Kernel dependencies and we want to leverage automatic schema versioning we have to implement `tigase.kernel.beans.RegistrarBean` in our class!

=== Configuration

Our class `TestRepositoryMDBean` is annotated with `@Bean` which sets its name as `repository` and sets parent as `TestComponent`. Instance of this component was configured by use under name of `test` in Tigase XMPP Server configuration file.
As a result, all configuration related to our repositories should be placed in `repository` section placed inside `test` section.

.Example
[source,dsl]
----
test(class: TestComponent) {
    repository () {
        // repository related configuration
    }
}
----

==== Defaults
As mentioned above, if we use `MDRepositoryBeanWithStatistics` as our base class for `TestRepositoryMDBean`, then we may have different data sources used for different domains.
By default, if we will not configure it otherwise, `MDRepositoryBeanWithStatistics` will create only single repository instance named `default`.
It will be used for all domains and it will, by default, use data source named the same as repository instance - it will use data source named `default`.
This defaults are equal to following configuration entered in the config file:
[source,dsl]
----
test(class: TestComponent) {
    repository () {
        default () {
            dataSourceName = 'default'
        }
    }
}
----

==== Changing data source used by repository
It is possible to make any repository use different data source than data source configured under the same name as repository instance.
To do so, you need to set `dataSourceName` property of repository instance to the name of data source which it should use.

.Example setting repository `default` to use data source named `test`
[source,dsl]
----
test(class: TestComponent) {
    repository () {
        default () {
            dataSourceName = 'test'
        }
    }
}
----

==== Configuring separate repository for domain
To configure repository instance to be used for particular domain, you need to define repository with the same name as domain for which it should be used.
It will, by default, use data source with name equal domain name.

.Separate repository for `example.com` using data source named `example.com`
[source,dsl]
----
dataSource () {
    // configuration of data sources here is not complete
    default () {
        uri = "jdbc:derby:/database"
    }
    'example.com' () {
        uri = "jdbc:derby/example"
    }
}

test(class: TestComponent) {
    repository () {
        default () {
        }
        'example.com' () {
        }
    }
}
----

.Separate repository for `example.com` using data source named `test`
[source,dsl]
----
dataSource () {
    // configuration of data sources here is not complete
    default () {
        uri = "jdbc:derby:/database"
    }
    'test' () {
        uri = "jdbc:derby/example"
    }
}

test(class: TestComponent) {
    repository () {
        default () {
        }
        'example.com' () {
            dataSourceName = 'test'
        }
    }
}
----

NOTE: In both examples presented above, for domains other than `example.com`, repository instance named `default` will be used and it will use data source named `default`.

=== Repository Versioning

It's also possible to enable repository versioning capabilities when creating custom implementation. There are a couple of parts/steps to fully take advantage of this mechanism.

Each `DataSource` has a table `tig_schema_versions` which contains information about component schema version installed in the database associated with particular DataSource.

==== Enabling version checking in implementation

First of all, repository implementation should implement `tigase.db.util.RepositoryVersionAware` interface (all it's methods are defined by default) and annotate it with `tigase.db.Repository.SchemaId`. For example
.Repository annoted with `SchemaId` and implementing `RepositoryVersionAware`
[source,java]
----
@Repository.SchemaId(id = "test-component", name = "Test Component")
public static class TestRepositoryMDBean extends MDRepositoryBeanWithStatistics<TestRepositoryIfc>
    implements TestRepositoryIfc {
…
}
----

This action alone will result in performing the check during Tigase XMPP Server startup and initialisation of repository whether tables, indexes, stored procedures and other elements are present in the configured data source in the required version.
By default, required version matches the implementation version (obtained via call to `java.lang.Package.getImplementationVersion()`), however it's possible to specify required version manually, either:

- by utilizing `tigase.db.util.RepositoryVersionAware.SchemaVersion` annotation:
[source,java]
----
@Repository.SchemaId(id = "test_component", name = "Test Component")
@RepositoryVersionAware.SchemaVersion(version = "0.0.1")
public static class TestRepositoryMDBean extends MDRepositoryBeanWithStatistics<TestRepositoryIfc>
    implements TestRepositoryIfc {
…
}
----
- or by overriding `tigase.db.util.RepositoryVersionAware.getVersion` method:
[source,java]
----
	@Override
	public Version getVersion() {
		return "0.0.1";
	}
----

==== Handling wrong version and the upgrade

To detect that version information in database is inadequate following logic will take place:

* if there is no version information in the database the service will be stopped completely prompting to install the schema (either via `update-schema` or `install-schema` depending on user preference);
* if there is an information about loaded component schema version in the repository and the base part of the required schema version (i.e. taking into account only _major.minor.bugfix_ part) is different from the one present in the repository then:
** if the required version of the component schema is _final_ (i.e. non `SNAPSHOT`) the server will shutdown and print in the log file (namely `logs/tigase-console.log`) terminal error forcing the user to upgrade the schema;
** if the required version of the component schema is _non-final_ (i.e. having `SNAPSHOT` part) then there will be a warning printed in the log file (namely `logs/tigase-console.log`) prompting user to run the upgrade procedure due to possible changes in the schema but the _server *will not* stop_;

Upgrade of the loaded schema in the database will be performed by executing:
[source,bash]
----
./scripts/tigase.sh upgrade-schema etc/tigase.conf
----

The above command will load current configuration, information about all configured data sources and enabled components, and then perform upgrade of the schema of each configured component in the appropriate data source.

Depending on the type of the database (or specified annotation), how the upgrade procedure is handled internally is slightly different.

[#update-relational]
===== Relational databases (external handling)

For all relational databases (MySQL, PostgreSQL, MS SQL Server, etc…) we highly recommend storing complete database schema in external files with following naming convention: `<database_type>-<component_name>-<version>.sql`, for example complete schema for our Test component version 0.0.5 intended for MySQL would be stored in file named `mysql-test-0.0.5.sql`. What's more - schema files must be stored under `database/` subdirectory in Tigase XMPP Server installation directory.

NOTE: this can be controlled with `external` property of `Repository.SchemaId` annotation, which defaults to "true", if set to `false` then handling will be done as described in <<update-relational>>

For example:

* `database/mysql-test-0.0.1.sql`
* `database/mysql-test-0.0.2.sql`
* `database/mysql-test-0.0.3.sql`
* `database/mysql-test-0.0.4.sql`
* `database/mysql-test-0.0.5.sql`

During the upgrade process all required schema files will be loaded in the ascending version order. Version range will depend on the conditions and will follow simple rules:

* Start of the range will start at the next version to the one currently loaded in the database (e.g. if the current version loaded to the database is `0.0.3` and we are deploying component version `0.0.5` then SchemaLoader will try to load schema from files: `database/mysql-test-0.0.4.sql` and `database/mysql-test-0.0.5.sql`)
* If we are trying to deploy a _SNAPSTHOT_ version of the component then schema file matching that version will always be included in the list of files to be loaded (e.g. if we are trying to deploy a nightly build with component version `0.0.5-SNAPSHOT` and currently loaded schema version in the database is `0.0.5` then SchemaLoader will include `database/mysql-test-0.0.5.sql` in the list of files to be loaded)

It's also possible to skip above filtering logic and force loading all schema files for particular component/database from `database/` directory by appending `--forceReloadAllSchemaFiles=true` parameter to the `upgrade-schema`/`install-schema` command.

===== Non-relational databases (internal handling)

If there is a need to handle database schema internally (for example for cases like NoSQL databases or simply there is such preference) then it's possible to do so by setting `external` attribute of `Repository.SchemaId` annotation to `false`:

[source,java]
----
@Repository.SchemaId(id = "test_component", name = "Test Component", external = false)
----

In such case, `updateSchema` method from `tigase.db.util.RepositoryVersionAware` interface should be implemented to handle installation/updating of the schema. It takes two arguments:

* `Optional<Version> oldVersion` - indicating current version of the schema loaded to the database (if it's present)
* `Version newVersion` - indicating required version (either version of component or specific version of the repository)

==== Setting required repository version in database

Each versioned schema file should consist at the end code responsible for setting appropriate version of the loaded schema in the form of Stored Procedure call with the name of the component and the version as parameters:

* Postgresql
[source,sql]
----
-- QUERY START:
select TigSetComponentVersion('test_component', '0.0.5');
-- QUERY END:
----
* MsSQL Server
[source,sql]
----
-- QUERY START:
exec TigSetComponentVersion 'test_component', '0.0.5';
-- QUERY END:
GO
----
* MySQL
[source,sql]
----
-- QUERY START:
call TigSetComponentVersion('test_component', '0.0.5');
-- QUERY END:
----
* Derby
[source,sql]
----
-- QUERY START:
call TigSetComponentVersion('test_component', '0.0.5');
-- QUERY END:
----

In case of schema handled internally, after successful load (i.e. execution of the implemented `tigase.db.util.RepositoryVersionAware.updateSchema` method returning `tigase.db.util.SchemaLoader.Result.ok`) the version in the database will be set to the current version of the component.

This allows (in case of schema handled externally) to load it by hand by directly importing `.sql` files into database.