Cloud Hot Folders: Migration, Configuration, and Troubleshooting

On-premise SAP Commerce projects moved files with SFTP drops and NFS mounts into hot folders. CCv2 has no filesystem to mount: Cloud Hot Folders reimplement the pattern on Azure Blob Storage, with a Spring Integration pipeline polling the container and feeding the same batch ImpEx machinery you already know. The concept translates cleanly; the configuration, connectivity, and failure modes do not, and this guide covers all three, consolidating two CX Works articles into one reference.

How the Pipeline Works#

Files land in a blob container (named hybris, path convention hybris/<tenant>/hotfolder/...; a container with any other name is invisible to the machinery). A polling adapter on the background processing aspect synchronizes blobs to local disk, moves them through the lifecycle folders, and hands them to the batch import pipeline:

hotfolder/           <- drop zone
hotfolder/processing <- picked up, being worked
hotfolder/archive    <- import succeeded
hotfolder/error      <- import failed

Each file walks the classic batch steps, visible in the logs: DOWNLOADED, HEADER_SETUP, HEADER_INIT, HEADER_TRANSFORMED, HEADER_EXECUTED, HEADER_CLEANUP. The transformation from CSV to ImpEx via converter mappings is unchanged from legacy hot folders, which is why migrations preserve converters untouched and rework only the transport layer.

The extensions involved: azurecloud, cloudcommons, cloudhotfolder, azurecloudhotfolder. After adding them, run a system update with all four selected, or the pipeline's types will not exist. Note also that the Azure connectivity underneath moved to the Azure Java SDK 12.x line (the old SDK 8.x was removed from the platform in 2025); if you carry very old custom code constructing AzureBlobSession directly, check it against the current constructors.

Local Development with Azurite#

You cannot get a per-developer Azure account from CCv2, and you do not need one: Azurite (the local blob emulator) stands in fully. The well-known development connection string in local.properties:

azure.hotfolder.storage.account.connection-string=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;
azure.hotfolder.storage.account.name=devstoreaccount1
azure.hotfolder.storage.container.name=hybris
azure.hotfolder.storage.container.create=true
azure.hotfolder.storage.container.hotfolder=${tenantId}/hotfolder
azure.hotfolder.storage.container.match.pattern=^((?!ignore).)*$

# required or nothing polls
cluster.node.groups=integration,yHotfolderCandidate

Drop a product-01.csv into hybris/master/hotfolder/ via Azure Storage Explorer attached to the local emulator, and the Azurite console narrates the lifecycle (PUT into processing/, DELETE of the original, PUT into archive/ with a timestamp suffix). If instead you see nothing at all, jump to the troubleshooting section; the causes are enumerable.

The default channel's behaviour is driven by properties (values here from a project called ABC):

cloud.hotfolder.default.mapping.root.dir=ABC
cloud.hotfolder.default.mapping.header.catalog=ABCMasterProductCatalog
cloud.hotfolder.default.mapping.header.net=false
cloud.hotfolder.default.mapping.file.name.pattern=^(customer|product|url_media)-\\d+.*

# processing order: essential first, media last
cloud.hotfolder.storage.file.sort.name.prefix.priority=coredata,sampledata,product,url_media
cloud.hotfolder.storage.file.sort.name.sequence=^(?<filename>.*)-(?<sequence>\\d*)(?<extension>.*)$

Migrating Legacy Hot Folder Configurations#

Legacy hot folders are file:inbound-channel-adapter beans watching directories. Migration has three escalating patterns, and most projects need all three at least once.

Pattern 1: File-name routing onto the default channel#

When the legacy folder existed only to apply a specific HeaderSetupTask (different catalog, different defaults), you often do not need a folder at all: extend the accepted file pattern and map names to a channel. The mapping is a MethodInvokingFactoryBean registering a regex against hotfolderInboundFileChannelMappings:

<bean class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
  <property name="targetObject" ref="hotfolderInboundFileChannelMappings"/>
  <property name="targetMethod" value="put"/>
  <property name="arguments">
    <list>
      <bean class="java.util.regex.Pattern" factory-method="compile">
        <constructor-arg value="^(reviews)-\d+.*"/>
      </bean>
      <ref bean="abcReviewsProcChannel"/>
    </list>
  </property>
</bean>

<int:channel id="abcReviewsProcChannel"/>

<int:service-activator input-channel="abcReviewsProcChannel"
                       output-channel="batchFilesHeaderInit"
                       ref="abcReviewsHeaderSetupTask" method="execute"/>

<bean id="abcReviewsHeaderSetupTask"
      class="de.hybris.platform.acceleratorservices.dataimport.batch.task.HeaderSetupTask">
  <property name="catalog" value="ABCProductCatalogUS"/>
  <property name="net" value="${cloud.hotfolder.default.mapping.header.net}"/>
  <property name="storeBaseDirectory" ref="baseDirectoryCloudHotFolder"/>
</bean>

Remember to widen cloud.hotfolder.default.mapping.file.name.pattern to accept the new prefix, or the default channel drops the file before your mapping ever sees it.

Pattern 2: A dedicated blob subfolder#

When the integration contract requires its own directory (.../hotfolder/reviews/), you replicate the trio the platform uses internally: an AzureBlobInboundSynchronizer (which subfolder, where processing/archive live), an AzureBlobSynchronizingMessageSource (blob-to-local sync), and an inbound channel adapter wiring it to the pipeline with transactional routing to archive or error:

<bean id="abcReviewsSynchronizer"
      class="de.hybris.platform.cloud.azure.hotfolder.remote.inbound.AzureBlobInboundSynchronizer">
  <constructor-arg name="sessionFactory" ref="azureBlobSessionFactory"/>
  <property name="remoteDirectory" value="#{azureHotfolderRemotePath}/reviews"/>
  <property name="moveToRemoteDirectory" value="#{azureHotfolderRemotePath}/reviews/processing"/>
  <property name="deleteRemoteFiles" value="${azure.hotfolder.storage.delete.remote.files}"/>
  <property name="filter" ref="azureHotfolderFileFilter"/>
  <property name="comparator" ref="azureHotFolderFileComparator"/>
</bean>

<bean id="abcReviewsMessageSource"
      class="de.hybris.platform.cloud.azure.hotfolder.remote.inbound.AzureBlobSynchronizingMessageSource">
  <constructor-arg name="synchronizer" ref="abcReviewsSynchronizer"/>
  <property name="autoCreateLocalDirectory" value="true"/>
  <property name="localDirectory" value="#{azureHotfolderLocalDirectoryBase}/#{azureHotfolderRemotePath}"/>
  <property name="maxFetchSize" value="${azure.hotfolder.storage.polling.fetch.batch-size}"/>
</bean>

<int:inbound-channel-adapter id="abcReviewsInboundAdapter"
    auto-startup="false" role="${cloud.hotfolder.storage.services.role}" phase="50"
    ref="abcReviewsMessageSource" channel="hotfolderInboundFileHeaderEnricherChannel">
  <int:poller fixed-rate="${azure.hotfolder.storage.polling.fixed.rate}"
              task-executor="azureChannelAdapterTaskExecutor"
              max-messages-per-poll="${azure.hotfolder.storage.polling.fetch.batch-size}">
    <int:transactional synchronization-factory="abcReviewsSyncFactory"
                       transaction-manager="azurePsuedoTxManager"/>
  </int:poller>
</int:inbound-channel-adapter>

<int:transaction-synchronization-factory id="abcReviewsSyncFactory">
  <int:after-commit channel="abcReviewsArchiveAdapter"/>
  <int:after-rollback channel="abcReviewsErrorAdapter"/>
</int:transaction-synchronization-factory>

plus two abstractAzureMoveMessageHandler children pointing at reviews/archive and reviews/error. Verbose, yes; but it is a mechanical translation, and the auto-startup="false" with role is not boilerplate to trim: it is what ties the adapter to the leadership election so exactly one node polls (more below).

Pattern 3: Custom processing order#

Legacy FileOrderComparator priority maps become a NamePrefixComparator wrapped for blob use and chained with the standard comparators:

<bean id="abcCameraPrefixComparator"
      class="de.hybris.platform.cloud.commons.spring.integration.file.comparators.NamePrefixComparator">
  <constructor-arg name="namePrefixPriority" value="manufacturer,model,lens,tripod,flash"/>
</bean>
<bean id="abcCameraBlobComparator"
      class="de.hybris.platform.cloud.azure.hotfolder.remote.file.comparators.AzureBlobNameComparatorAdapter">
  <constructor-arg name="comparator" ref="abcCameraPrefixComparator"/>
</bean>
<util:list id="abcCameraComparatorList">
  <ref bean="abcCameraBlobComparator"/>
  <ref bean="azureHotFolderFileNameComparator"/>
  <ref bean="azureHotFolderFileNameSequenceComparator"/>
  <ref bean="azureHotFolderFileModifiedComparator"/>
</util:list>
<bean id="abcCameraComparator"
      class="org.apache.commons.collections4.comparators.ComparatorChain">
  <constructor-arg ref="abcCameraComparatorList"/>
</bean>

Reference the chain from that channel's synchronizer, and manufacturers import before models before lenses, exactly as the dependency order demands.

Getting Files In: Connectivity Options#

SFTP and NFS are gone; the counterpart systems need a new way to reach the blob container. The menu, with the honest trade-offs:

A working Azure CLI skeleton (credentials come from the Cloud Portal, visible to authorized users only):

#!/bin/bash
export AZURE_STORAGE_ACCOUNT=<storage_account_name>
export AZURE_STORAGE_KEY=<storage_account_key>

az storage blob upload --container-name hybris \
   --file product-$(date +%s).csv \
   --name master/hotfolder/product-01.csv

az storage blob list --container-name hybris --output table

Whatever the channel, the naming contract stands: files must land under hybris/master/hotfolder/... and match the configured name patterns, or they will sit there forever, unprocessed and unexplained.

Troubleshooting: The File That Would Not Process#

The support tickets all read the same: "we dropped the file and nothing happened." Work this list top to bottom.

1. Are you on the right node? Hot folder processing runs on the background processing aspect only. Logs from storefront nodes will never show hot folder activity.

2. Is the connection string actually valid? The property is azure.hotfolder.storage.account.connection-string, and there is a specific CCv2 gotcha: deployment-injected strings sometimes carry EndpointSuffix=core.windows.net where the hot folder machinery needs an explicit BlobEndpoint= to connect. Symptom: the container never gets auto-created at startup, with a StorageException/IOException if you raise logging. Verify quickly from a local workspace by pointing your local.properties at the same string with a distinctive azure.hotfolder.storage.container.name; if the container appears on startup, the string works. Fix it per environment in Cloud Portal under the Background Processing service's properties (the pod restarts on save), then confirm in HAC that the effective value changed.

3. Are the node groups set? The background processing aspect needs, in manifest.json:

{
  "name": "backgroundProcessing",
  "properties": [
    { "key": "cluster.node.groups", "value": "integration,yHotfolderCandidate" }
  ]
}

integration starts the Spring Integration thread pools; yHotfolderCandidate enters the node into leadership election, and the elected leader starts the hot folder services. Set on no aspect: nothing polls. Set on multiple aspects: nodes compete for files. Verify the effective value in HAC on each node.

4. Is the polling thread alive? Take a thread dump (HAC, Monitoring, Thread Dump) on the background processing node and search for AzureIntegrationTaskExecutorThread. No thread, no processing. Two known killers:

• A rejected-execution policy someone changed. The azureChannelAdapterTaskExecutor must use ThreadPoolExecutor$CallerRunsPolicy; with DiscardPolicy, the polling task gets silently discarded under load and never comes back.
• A stale ApplicationResourceLock. If a node died holding the hot folder leadership lock, the replacement cannot acquire it and the thread is never created. Check and clear it (note the FlexibleSearch quirk where the row sometimes only shows via raw SQL):

SELECT * FROM applicationresourcelock;
DELETE FROM applicationresourcelock WHERE p_lockkey = '<key-from-select>';

The lock re-acquires immediately, no restart needed, and files start moving.

5. Is a zero-byte file poisoning the folder? A zero-byte blob (some FTP clients create marker files) throws a StorageException during transfer, and the exception handling in AzureBlobInboundSynchronizer.transferFilesFromRemoteToLocal() stops processing of the remaining files in that folder too. One empty file, whole folder stalled. Delete it, and prevent recurrence with the match pattern (azure.hotfolder.storage.container.match.pattern) or by fixing the sender.

6. Only then, look at the content. If files reach error/, the transport is fine and the problem is the batch import itself: converter mappings, ImpEx validity, catalog names. That is ordinary import debugging, with the failed file conveniently preserved for replay.

Operational Notes#

• Blob storage counts against the CCv2 storage quota; archive folders grow forever unless you schedule cleanup of processed files.
• Production background processing runs at least two nodes; the leadership election means exactly one polls at a time and failover is automatic. Do not "fix" perceived idleness on the second node.
• Monitor the error/ folder and alert on arrivals; an unmonitored error folder is where integration incidents go to age quietly.
• Keep converter mappings and channel configs in the repository with tests that run a sample file through the transformation. The pipeline is Spring config all the way down, and Spring config without tests rots.

Cloud Hot Folders are one of those CCv2 subsystems that are completely reliable once correctly configured and completely opaque until then. The configuration is this guide; bookmark the troubleshooting list for the day a file does not move.