Block Types and CIF Data

Basics of Block Types Data

Almost any block type requires some configuration, which is persisted in a database table. The name of the table is defined in the $btTable property of the block controller, and the table structure is defined in the db.xml file.

Let's assume, for example, that your block type needs an image (represented by the numeric ID of a Concrete file) stored in a field named imageFileID, and a Rich Text stored in a field named imageDescription.

The db.xml file can be something like this:

<?xml version="1.0" encoding="UTF-8"?>
<schema
    xmlns="http://www.concrete5.org/doctrine-xml/0.5"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.concrete5.org/doctrine-xml/0.5 https://concretecms.github.io/doctrine-xml/doctrine-xml-0.5.xsd"
>
    <table name="btMyBlockType">
        <!-- bID is required and managed by Concrete -->
        <field name="bID" type="integer">
            <unsigned />
            <key />
        </field>
        <field name="imageFileID" type="integer">
            <unsigned />
            <notnull />
        </field>
        <field name="imageDescription" type="text">
            <notnull />
        </field>
    </table>
</schema>

In your block controller, you may want to define these two fields (it's highly recommended for PHP 8 compatibility):

    /**
     * @var int|null it's null when users are adding a new block instance
     */
    protected $imageFileID;
    /**
     * @var string|null it's null when users are adding a new block instance
     */
    protected $imageDescription;

When users add a new block, you may want to initialize those values for the view:

    public function add()
    {
        $this->set('imageFileID', null);
        $this->set('imageDescription', '');
    }

When users instead edit an existing block, we have to prepare the imageDescription field for the Rich Text editor using the translateFromEditMode() method of the LinkAbstractor. We must also use the translateFrom() method of the LinkAbstractor when preparing the view of the Rich Text for website visitors.

    public function edit()
    {
        $this->set('imageDescription', \Concrete\Core\Editor\LinkAbstractor::translateFromEditMode($this->imageDescription));
    }

    public function view()
    {
        $this->set('imageDescription', \Concrete\Core\Editor\LinkAbstractor::translateFrom($this->imageDescription));
    }

When users save the block, Concrete invokes the save() method of your block controller. This method receives an array with all the values submitted through the web interface form. The default implementation of save() retrieves the database table's field names and constructs a row to be saved in the database, using values from the array that match the table's field names. Since imageDescription contains Rich Text, we must convert the received value into a format suitable for storage. That's achieved by calling the translateTo() method of the LinkAbstractor, so we have to implement our own save() method, and we must be sure that the received value of imageFileID is an integer:

    public function save($args)
    {
        $args['imageFileID'] = (int) ($args['imageFileID'] ?: 0);
        $args['imageDescription'] = \Concrete\Core\Editor\LinkAbstractor::translateTo($args['imageDescription'] ?? '');
        parent::save($args);
    }

What do all those LinkAbstractor calls do? For example, they convert links to other pages between different formats.

Let's assume your Rich Text contains a link to the page with ID 12 available at the path /blog.

In the Rich Text editor, links are created in this format:

<a href="https://www.yourwebsite.com/index.php?cID=12">Visit our blog</a>

In the database, we need to store links without domain dependencies:

<a href="{CCM:CID_12}">Visit our blog</a>

When rendering the block for website visitors, we get:

<a href="https://www.yourwebsite.com/blog">Visit our blog</a>

That's all you need to make the block type work in a single Concrete installation.

However, additional work is required for users that want to copy data from website A to website B (e.g., using the Migration Tool or Blocks Cloner). These migrations use the CIF format (a text file in XML format). The process involves extracting data from website A in CIF format and importing it into website B. CIF is also used when installing Concrete itself and by many themes with the Full Content Swap feature.

Therefore, it is crucial that your block type correctly supports exporting and importing CIF data.

Exporting/Importing Block Data to/from CIF

Let's assume that website A has a page with ID 12 at /blog and a file with ID 34 named flowers.png. When recreating /blog in website B and uploading flowers.png, their IDs will likely be different (e.g., /blog might be assigned ID 56, and flowers.png ID 78).

Thus, we cannot simply use website A's IDs when exporting and importing imageFileID and imageDescription fields. We need to convert these values into a format understandable by both websites.

How can this be achieved?

When generating CIF data from a block instance, Concrete calls the export() method of the block controller. If not overridden, the default implementation generates an XML element for every table field, setting element values to those read from the database.

For example, if you do nothing, your block data will be exported as:

<block type="my_block_type">
    <data table="btMyBlockType">
        <record>
            <imageFileID>34</imageFileID>
            <imageDescription><![CDATA[
                <a href="{CCM:CID_12}">Visit our blog</a>
            ]]></imageDescription>
        </record>
    </data>
</block>

Since those IDs are specific to website A, they won't work for website B.

We need to convert the links contained in Rich Texts, as well as to convert the IDs of files (and other things like page IDs, and so on), both when exporting and when importing data.

Converting IDs

For converting the IDs, it's enough to define these properties in your block type controller:

• for file IDs: use $btExportFileColumns In our example, the block controller should have: php protected $btExportFileColumns = ['imageFileID'];
• for page IDs: use $btExportPageColumns
• for page type IDs: use $btExportPageTypeColumns
• for page feed IDs: use $btExportPageFeedColumns
• for file folder IDs: use $btExportFileFolderColumns

In our example, since imageFileID is in the $btExportFileColumns property, Concrete will export the block data like this:

<block type="my_block_type">
    <data table="btMyBlockType">
        <record>
            <imageFileID>{ccm:export:file:123456789012:flowers.png}</imageFileID>
        </record>
    </data>
</block>

As you can see, Concrete replaces the exported value of imageFileID with a string that can be imported back into another website.

When Concrete in website B imports that XML, it will automatically convert {ccm:export:file:123456789012:flowers.png} to the ID of the file in website B, passing your save() method an array where imageFileID will have the value 78.

Converting Rich Texts

In the example above, if we don't do anything, we saw that the exported CIF will contain a value for the link that's specific to website A:

<block type="my_block_type">
    <data table="btMyBlockType">
        <record>
            <imageDescription><![CDATA[
                <a href="{CCM:CID_12}">Visit our blog</a>
            ]]></imageDescription>
        </record>
    </data>
</block>

Since the page with ID 12 only exists in website A, this won't work for website B (where the ID may be 56).

For Concrete 9.4.0 and Later

Since Concrete v9.4.0, you simply have to define the $btExportContentColumns property in your block type controller:

php protected $btExportContentColumns = ['imageDescription'];

By doing so, Concrete will then export the block data like this:

<block type="my_block_type">
    <data table="btMyBlockType">
        <record>
            <imageDescription><![CDATA[
                <a href="{ccm:export:page:/blog}">Visit our blog</a>
            ]]></imageDescription>
        </record>
    </data>
</block>

As you can see, Concrete replaces the exported value of imageDescription with a string that can be imported back into another website.

For Concrete from 9.2.1 to the latest 9.3

For versions older than 9.4.0 but at least 9.2.1, Concrete will correctly import rich texts with links like <a href="{ccm:export:page:/blog}">Visit our blog</a>, but it cannot export them, so we have to handle this manually.

This can be achieved by overriding the export() method with something like this:

    /**
     * {@inheritdoc}
     *
     * @see \Concrete\Core\Block\BlockController::export()
     */
    protected function export(\SimpleXMLElement $blockNode)
    {
        parent::export($blockNode);
        // Let's take the value stored in the imageDescription element of <block><data><record><imageDescription>...</imageDescription></record></data></block>
        $imageDescription = (string) $blockNode->data->record->imageDescription;
        if ($imageDescription === '') {
            // Nothing to do
            return;
        }
        // Let's remove imageDescription element
        unset($blockNode->data->record->imageDescription);
        // Let's fix the value of $imageDescription
        $imageDescriptionFixed = \Concrete\Core\Editor\LinkAbstractor::export($imageDescription);
        // Let's store the value of $imageDescriptionFixed in the XML
        $xmlService = $this->app->make(\Concrete\Core\Utility\Service\Xml::class);
        $xmlService->createCDataNode($blockNode->data->record, 'imageDescription', $imageDescriptionFixed);
    }

For Concrete older than 9.2.1

For versions older than 9.2.1, in addition to fixing the export step (overriding the export() method as described above), you also need to fix the import step.

This can be achieved by overriding the getImportData() method with something like this:

    /**
     * {@inheritdoc}
     *
     * @see \Concrete\Core\Block\BlockController::getImportData()
     */
    protected function getImportData($blockNode, $page)
    {
        $args = parent::getImportData($blockNode, $page);
        if (!empty($args['imageDescription'])) {
            $args['imageDescription'] = \Concrete\Core\Editor\LinkAbstractor::import($args['imageDescription']);
        }

        return $args;
    }

With this change, your save() method will receive a value suitable for website B, referencing the page with ID 56.

Custom Processing

You may have some custom logic in the save() method of your block type controller. For example, you may parse the fields received when users save the block via the web interface, and do some processing on them before storing the fields in the database (like deriving new database fields, or storing data in JSON format).

This may be problematic when your save() method is called while importing data from CIF files.

To fix this issue, you can do one or more of the following:

• pre-process the data read from the CIF files by overriding the getImportData() method, formatting the data into a suitable format accepted by your save() method
• update your save() method so that accepts data both from the web form and from CIF files
• change the data exported to CIF format by overriding the export() method

Example of custom processing: JSON data

Let's assume your block type is an image slider, and you save each slides (represented by an image file ID and a caption) in a JSON array.

When users add or edit a block via the web interface, your save() may receive an array derived from the POSTed data like this:

[
    'imageFileID' => [
        '1',
        '2',
        '3',
    ],
    'caption' => [
        'Caption 1',
        'Caption 2',
        'Caption 3',
    ],
]

your save() method could then serialize the fields received in JSON format, to be stored in a database field named slides, with something like this:

public function save($args)
{
    $slides = [];
    if (isset($args['imageFileID'])) {
        for ($i = 0; ; $i++) {
            if (!array_key_exists($i, $args['imageFileID'])) {
                break;
            }
            $slides[] = [
                'imageFileID' => (int) $args['imageFileID'][$i],
                'caption' => $args['caption'][$i] ?? '',
            ];
        }
    }

    return [
        'slides' => json_encode($slides),
    ];
}

By default, when exporting data to CIF, Concrete would generate an XML like this (indented to make it clearer):

<block type="my_block_type">
    <data table="btMyBlockType">
        <record>
            <slides>
                [
                    {"imageFileID":1,"caption":"Caption 1"},
                    {"imageFileID":2,"caption":"Caption 2"},
                    {"imageFileID":3,"caption":"Caption 3"}
                ]
            </slides>
        </record>
    </data>
</block>

Since the file IDs can't be exported as is to another website, we have to fix the values of the imageFileID fields in the JSON. This can be done by overriding the export() method like this.

public function export(\SimpleXMLElement $blockNode)
{
    $jsonSlides = (string) $this->slides;
    if ($jsonSlides !== '') {
        $slides = json_decode($jsonSlides);
        if (array($slides)) {
            foreach ($slides as $slideIndex => $slide) {
                $slide->imageFileID = \Concrete\Core\Backup\ContentExporter::replaceFileWithPlaceHolder($slide->imageFileID);
            }
            $jsonSlides = json_encode($slides, JSON_UNESCAPED_SLASHES);
        }
    }
    $xmlService = $this->app->make(\Concrete\Core\Utility\Service\Xml::class);
    $dataNode = $blockNode->addChild('data');
    $dataNode->addAttribute('table', $this->getBlockTypeDatabaseTable());
    $recordNode = $dataNode->addChild('record');
    $xmlService->createCDataNode($recordNode, 'slides', $jsonSlides);
}

That way, wne the block gets exported, we'll have an XML like this (indented to make it clearer):

<block type="my_block_type">
    <data table="btMyBlockType">
        <record>
            <slides>
                [
                    {"imageFileID":"{ccm:export:file:123456789012:imageA.png}","caption":"Caption 1"},
                    {"imageFileID":"{ccm:export:file:123456789013:imageB.png}","caption":"Caption 2"},
                    {"imageFileID":"{ccm:export:file:123456789014:imageC.png}","caption":"Caption 3"}
                ]
            </slides>
        </record>
    </data>
</block>

As you can see, the JSON fields imageFileID now contain a file representation that doesn't depend of the ID of files, and it can be understood by other Concrete websites.

Now the problem is that, when importing back the block from that XML, the data your save() method will receive will be an array, with just one entry, named slides (that's the name of the XML field), containing the JSON slides with that exportable format for images.

You have two options now: you can adjust the data read from CIF before it's sent to save(), or you can change save() so that it accepts this new format of data.

Let's implement first approach, by overriding the getImportData() method like this:

/**
 * @var \SimpleXMLElement $blockNode
 * @var \Concrete\Core\Page\Page $page
 *
 * @return array
 */
protected function getImportData($blockNode, $page)
{
    // First, let's extract the data from CIF using the default Concrete implementation of getImportData() 
    $args = parent::getImportData($blockNode, $page);
    // Next, let's prepare the data for the save() method:
    $result = [
        'imageFileID' => [],
        'caption' => [],
    ];
    $jsonSlides = $args['slides'] ?? ''; 
    if (!empty($args['slides'] ?? '')) {
        $slides = json_decode($args['slides']);
        if (is_array ($slides)) {
            $inspector = $this->app->make('import/value_inspector');
            foreach ($slides as $slide) {
                // This line converts strings like '{ccm:export:file:123456789012:imageA.png}' to the file ID
                $result['imageFileID'][] = inspector->inspect((string) $slide->imageFileID)->getReplacedValue();
                $result['caption'][] = (string) $slide->caption;
            }
        }
    }

    return $result;
}

That way, the save() method will receive the value of the $result generated by your getImportData() method.