Release Notes

What's New for Elastic Path Commerce 7.1.0

What's New for Elastic Path Commerce 7.1.0

Release Highlights

Elastic Path Commerce 7.1.0 includes performance and scalability improvements, report conversions, maturing of the Helix programming model, and support for new databases. The highlights of the release are:

  • Performance and scalability: Significant improvements were made in the following areas:
    • Cortex throughput and response time
    • System stability under load
    • Handling of larger shopping carts
    • Helix advise performance

    In addition, support for configuring and deploying per-environment cache settings was implemented.

  • Reporting: The reporting framework and the following four reports were converted from the desktop Commerce Manager to the web: Order Status, Order Summary, Returns and Exchanges, and Shopping Cart Promotion Usage reports. Additional reports will be converted in future releases.
  • Helix maturity: The majority of Cortex resources were converted to the Helix programming model and multiple refinements were made to the programming model.
    • The /rates, /slots, /stocks and /subscriptions resources were removed.
    • All other resources were converted with the exception of the /items, /payments and /events resources.
    • Multiple changes were made to structured messages, including adding a type field, converting all needinfo links to structured messages, returning new structured messages via Helix advisors, and refactoring id and field-name values to improve consistency.
    • Programming model refinements were made to strengthen validation and reduce repetitive code.
  • Database support: Support was added for MySQL 5.7 and AWS Aurora databases.
  • Standalone demo package: A standalone demo package can now be created from any project using the packager module.
  • Email attachment support: Support was added for adding attachments to email messages that are sent via JMS.

System Requirements and Compatibility

The system requirements are documented at System Requirements.

Elastic Path Commerce 7.1.0 is compatible with the following:

Elastic Path Component Version Location Branch or Filename
Elastic Path CloudOps for AWS 2.1.x code.elasticpath.com release/2.1.x
Elastic Path Commerce Demo Package 7.1.0 support.elasticpath.com EP-Commerce-Demo-7.1.0.zip
Elastic Path Commerce for Adobe Marketing Cloud 1.6 support.elasticpath.com EP-Commerce-for-AMC-1.6.0.zip

Functional Changes

Core Commerce

  • PB-2271 - Added support for attachments to email messages that are sent via JMS.
  • PB-3001 - Added ability to create a standalone demo package. See the upgrade notes below for details.
  • PB-3116 - Improved support for per-environment configuration. See the upgrade notes below for details.
Cortex API
  • PB-2238 - Added a type field to structured messages to help API clients decide how to handle different message conditions. See the upgrade notes below for details.
  • PB-3000 - When reading a cart, cart.item.not.available structured messages are now returned for items that are not available, and the purchase form is blocked.
  • PB-3011 - Refactored structured message id and field-name values for consistency, removed unnecessary messages, and added new messages via Helix advisors. See the upgrade notes below for details.
  • PB-3072 - Added a /lookups resource to the Cortex root.
Commerce Manager
  • [Multiple] - Converted the reporting infrastructure and an initial set of reports to the web Commerce Manager. See the upgrade notes below for details.
  • PB-3114 - Added filtering of .csv files to the CSV Import file upload dialog.

Development Changes

Core Commerce

  • PB-1129 - Solr search indexes are automatically rebuilt if they have been cleaned either during a build or a deploy.
  • PB-2237 - Added support for Amazon Aurora RDS database (MySQL 5.6 compatible).
  • PB-2517 - Upgraded Oracle JDK to version 8u144. See the upgrade notes below for details.
  • PB-2625 - The Import/Export tool now exports XML elements in a consistent order to simplify viewing and tracking of changes.
  • PB-3019 - Removed com.jcraft.jsch dependency from ep-core as it was not being used and was a potential security vulnerability.
  • PB-3095 - Moved Commerce Manager log4j.properties to ext-cm-webapp for consistency with other webapps and to make it easier for projects to customize logging.
  • PB-3124 - Added additional information to EpValidationException to help identify the root cause of the validation error.
  • PB-3181 - Upgraded MySQL JDBC driver to version 5.1.44. See the upgrade notes below for details.
  • PB-3181 - Added support for MySQL 5.7. This required renaming the VIRTUAL database column to IS_VIRTUAL. See the upgrade notes below for details.
  • PB-3182 - Upgraded Apache Tomcat to version 7.0.81. See the upgrade notes below for details.
  • PB-3227 - A parent POM was added to the source release package and the cortex-ep-integration module was renamed to cortex-resources. See the upgrade notes below for details.
  • PB-3341 - Added check to system smoke tests to verify Cortex webapp startup.
  • PB-3348 - Added a new search server health check at /<context>/indexstatus to verify that all search indexes have been built on a clean deployment. This health check is intended for use in development CI pipelines and not in production.
  • PB-3352 - Updated external cache names to improve consistency and accuracy. See the upgrade notes below for details.
Cortex API
  • [Multiple] - Converted multiple Cortex resources to the Helix programming model. See the upgrade notes below for details.
  • PB-2059 - Changed the Helix validator NoOrphanedResourceRule from an error to a warning.
  • PB-2194 - Validated that constructor injection in Helix is done only on public constructors.
  • PB-2750 - Included Cortex API definitions in the source release package.
  • PB-2966 - Implemented stronger typing of URI identifier parts. See the upgrade notes below for details.
  • PB-2978 - Upgraded RxJava to version 2.1.0. This is major version upgrade that requires code changes to custom Helix resources. See upgrade notes below for details.
  • PB-2979 - Added a validation rule to ensure that @RequestForm injectors are used only when creating or updating resources.
  • PB-2981 - Added sample API definition and Helix resource modules to extensions/cortex to provide examples to follow when creating new Helix resources.
  • PB-2983 - Improved the error message when a Helix resource entity is constructed without using the builder.
  • PB-2987 - Added an AliasRepository interface to establish a standard pattern for getting default objects (e.g. default cart).
  • PB-2999 - Added a validation rule to ensure that @RequestIdentifier annotations in Helix prototypes is injecting types assignable from ResourceIdentifier.
  • PB-3014 - Added a validation rule to ensure that @Inject annotations in Helix prototypes have qualifiers (such as @RequestIdentifier, @RequestForm, @ResourceRespository)
  • PB-3041 - Removed the /slots resource. See Announcements and Notices 7.1.0 for more information.
  • PB-3100 - Added and fromNullableAsSingle methods to ReactiveAdaptor to simplify handling of service methods that return null domain objects.
  • PB-3112 - Added information to several Helix validator error messages to identify the injection point that failed.
  • PB-3139 - Added methods to ReactiveAdaptor to provide additional options for handling nulls.
  • PB-3250 - Enhanced Cortex health check to also validate database and JMS availability and to use the standard health check URL pattern of /cortex/status/[lb|info.json|info.html]. The existing Cortex /healthcheck resource was not modified and is still available.
  • PB-3357 - Deprecated the Select method in generated interfaces that extend ChoiceResourceDefinition. Use SelectWithResult instead to gain control of the returned status code.
  • PB-3358 - Deprecated the Submit method on generated interfaces that extend FormResourceDefinition. Use SubmitWithResult instead to gain control of the returned status code.
Commerce Manager
  • PB-2802 - Eliminated the cm-invoker and ext-cm-invoker modules so that builds can be done from the root POMs. This required source code restructuring that is described in the upgrade notes below.
  • PB-3088 - Added a Quartz job to cleanup temporary CSV export and report download files from Tomcat temp directory.
  • PB-3166 - Enabled Checkstyle/PMD for extensions/cm modules.

Performance Improvements

Core Commerce

  • PB-1593 - Implemented application caching of price list assignments to reduce database load.
  • PB-3069 - Implemented application caching of base amounts to reduce database load.
  • PB-3070 - Implemented application caching of promotion rules to reduce database load.
  • PB-3093 - Implemented application caching of bundle price adjustments to reduce database load.
  • PB-3099 - Optimized caching of settings with multiple values to read all values at the same time rather than individually.
  • PB-3110 - Implemented application caching of Groovy condition evaluations to reduce CPU usage.
  • PB-3111 - Added an index on TCUSTOMER.LAST_EDIT_DATE to eliminate full table scans by the customer search index builder.
  • PB-3212 - Eliminated checking whether an item has a price when adding to cart or merging carts. This is no longer necessary as a cart.item.not.available message is now returned when reading the cart, which blocks the purchase.
  • PB-3238 - Added a TORDERSKUPARENT join table to eliminate a full table scan on TORDERSKU whenever an order was read from the database. See the release notes below for details.
  • PB-3254 - Deferred shopping cart deletion to reduce checkout response times. The cart is marked inactive after an order is created and then deleted at a later time by the cleanupInactiveCartsJob.
  • PB-3255 - Added a batchLimit parameter to jpa-persistence.xml to group multiple insert and delete operations into a single SQL query.
  • PB-3257 - Combined writes to TTAXJOURNAL on checkout into a single transaction.
  • PB-3270 - Eliminated unnecessary cache lookups from PromotionRuleDelegate.cartContainsSku().
  • PB-3277 - Optimized retrieval of applied rules during checkout by combining multiple database queries into a single query.
  • PB-3301 - Increased the JPA @TableGenerator allocation size for high concurrency tables to reduce contention on the jpa_generated_keys table.
Cortex API
  • PB-3096 - Ensured that the Helix CachingResourceOperatorAnalyzerImpl is always used instead of the non-caching version. The non-caching version was being loaded in some environments which resulted in a major performance drop.
  • PB-3097 - Optimized reflection handling in Helix HandlerProxy classes to reduce CPU usage for link and advise operations.
  • PB-3128 - Eliminated string formatting in OperationResultCacheImpl to reduce CPU usage for link and advise operations.
  • PB-3152 - Moved injection of resourceServerName from UriBuilder classes to the corresponding UriBuilderFactory classes to reduce CPU usage.
  • PB-3161 - Replaced RxJava with Java streams in ResourceOperationLinker and ResourceOperationAdvisor to reduce CPU usage for link and advise operations.
  • PB-3183 - Optimized Helix resource locator logic to reduce CPU usage for link discovery.
  • PB-3248 - Bypassed unnecessary authorization and METHOD_NOT_ALLOWED cache checks to reduce CPU usage for link and advise operations.
  • PB-3263 - Upgraded Logback to 1.2.3 to reduce thread contention under load. This also required updating Slf4j to 1.7.25 (CUST-1250).
Commerce Manager
  • PB-3346 - Optimized database queries for Order Summary report.
  • PB-3347 - Optimized database queries for Order Status report.

Bug Fixes

Core Commerce

  • PB-3090 - The -Pdo-import and -Pdo-export Maven profiles in the ext-importexport-cli module did not successfully invoke the Import/Export tool.
  • PB-3115 - Database deadlocks were occurring under load because MySQL was deployed with the default transaction isolation level of REPEATABLE_READ. This was resolved by enforcing an isolation level of READ_COMMITTED in jpa-persistence.xml.
  • PB-3155 - The order total price for inclusive tax jurisdictions was not being rounded correctly and did not always match the sum of item prices (CUST-1238).
Cortex API
  • PB-1741 - Cortex Studio batch item lookup did not work with multiple item codes.
  • PB-2403 - Cortex API schema validation was not working when accessing API definitions in an IDE.
  • PB-2977 - The Fluent RelOS Client did not flush the logger when an exception or error occurred.
  • PB-3064 - Overriding Spring beans in the core-blueprint.xml was fixed by introducing an ext-core-bluprint.xml file into the ext-core module.
  • PB-3074 - Linking from a legacy resource to a Helix resource used resource name instead of entity-type, which worked only when resource name was the same as entity-type.
  • PB-3075 - Extended Helix entity builders did not accept an OOTB entity as a parameter.
  • PB-3229 - Adding an item with required configurable fields to a cart or wish list did not return an error if all configurable fields were empty.
  • PB-3262 - The cart line item list price field was showing the purchase price (CUST-1247).
  • PB-3280 - Changing the quantity of a cart line item was unpredictable when there was another item in the cart with the same SKU code but different configuration values.
  • PB-3285 - During cart merge, not-sold-separately dependent items were incorrectly removed from the cart when the parent item was also in the cart (CUST-971).
  • PB-3307 - Zooming into an item list that contains a bundle as well as the bundle's constituents as separate items did not zoom into the bundle's constituents correctly (CUST-1255).
  • PB-3316 - The Helix validator did not detect invalid nested identifiers when injecting a request identifier into a prototype.
  • PB-3355 - The oauthTokenExpiry.config file did not override the default OAuth token expiry timeout.
Commerce Manager
  • PB-2137 - Creation of a new CM user in the standalone demo package failed with a "Bound must be positive" error.
  • PB-2935 - CSV import of coupon codes failed when the file contained existing codes. It now ignores any existing codes, allowing the import to be re-run.
  • PB-2963 - Attempting to remove a product category assignment for a non-primary category failed with error "Unable to remove the primary category for this product".
  • PB-2972 - The Base Amount CSV import job had an incorrect title.
  • PB-3113 - Concurrent access of the Changeset Results tab caused an "Invalid thread access" error.
  • PB-3118 - OSGi BundleException errors were logged when shutting down the CM webapp.
  • PB-3121 - Updating multiple products via CSV import when change sets are enabled failed with an OpenJPA ArgumentException (CUST-1221).
  • PB-3135 - Changing quantity of bundle items caused a Null Pointer Exception.
  • PB-3174 - Users were not logged out after their session timed out.
  • PB-3266 - Executing a query in the Advanced Query Builder caused an "Invalid thread access" error.
  • PB-3282 - Changes to the payment gateway selection in the Store Payments tab were not saved.
  • PB-3330 - Automatically applying coupons during authentication failed in the CouponAutoApplyTransitionEventHandler.
  • PB-3338 - Searching for a customer phone number did not ignore spaces and separators.
  • PB-3353 - After session timeout the browser redirected to a blank page instead of the login page.

3rd Party Library Changes and Upgrades

Library New Version Previous Version
Apache Batik 1.7 1.6
Apache Tomcat 7.0.81 7.0.75
Eclipse BIRT 4.7.0 2.2.0
Eclipse RAP 3.2 3.1
Logback 1.2.3 1.1.3
MySQL JDBC Connector 5.1.44 5.1.37
Oracle JDK 8u144 8u66
RxJava 2.1.0 1.2.4
Slf4j 1.7.25 1.7.12

Upgrade Notes

The Upgrade Guide provides general advice on upgrading Elastic Path projects. Specific upgrade considerations for Core Commerce 7.1.0 and Cortex API 7.1.0 are covered below:

Converted initial set of Commerce Manager reports

The reporting infrastructure was upgraded to work with the web-based Commerce Manager, and the following initial set of reports were converted:

  • Order Status
  • Order Summary
  • Returns and Exchanges
  • Shopping Cart Promotion Usage

The converted reports are not exact replicas of the previous reports. Functional changes include:

  • The Order Status report no longer contains return details as these are available in the Returns and Exchanges report.
  • The Returns and Exchanges report has additional columns with return and exchange information and fewer columns relating to the original order.
  • Report parameters are remembered when switching between reports to simplify running multiple reports for the same store and time period.

Converted multiple Cortex resources to Helix programming model

The following resources were converted to the Helix programming model:
  • /addresses
  • /availabilities
  • /carts
  • /coupons
  • /discounts
  • /emails
  • /geographies
  • /lookups
  • /navigations
  • /orders
  • /prices
  • /promotions
  • /purchases
  • /recommendations
  • /searches
  • /shipments
  • /shipmentdetails
  • /taxes
  • /totals

When upgrading:

  • If you have extended these resources, you will need to convert your extension code to the Helix programming model.
  • The associated repositories were refactored to return RxJava results. This refactoring may impact custom resources that depend on these repositories.
  • If you have overridden permissions for these resources, you may need to modify permission parameters to use the new fully-qualified names listed at Cortex Authorization. For example, the cartId parameter was renamed to carts.cart-id
  • Be aware that many resource link URIs were modified during Helix conversions, but that the rel: names were not changed. As a best practice, you should not hard-code resource link URIs in your code.
  • Structured messages are returned by the converted resources for all business state errors, validation errors and needinfo conditions. If desired for backward compatibility, the Cortex webapp can be started with the -DneedInfoEnabled=true flag to return legacy needinfo links.
  • HTTP Status 409 Conflict is now returned by Helix resources when an InvalidBusinessStateException occurs. Previously Status 403 Forbidden was returned by some resources.

Added type field to Cortex structured messages

A type field was added to structured messages to help API clients decide how to handle different message conditions. Currently the system generates error and needinfo messages. In addition warning, information and promotion message types were defined for future use . See Types of Structured Messages for more information.

When upgrading:
  • Populate the type field for all custom structured messages.

Refactored Cortex structured messages

The following changes were made to structured messages:
  • Message id and field-name values were updated for consistency.
  • Messages that were not useful for user interactions were removed and are now returned as Status 500 errors.
  • New business state errors are returned by Helix advisors when reading and updating Cortex resources.
  • Structured messages are returned for all needinfo conditions. If desired for backward compatibility, the Cortex webapp can be started with the -DneedInfoEnabled=true flag to return legacy needinfo links.
When upgrading:
  • API clients will need to refactor their error and needinfo handling to reflect these changes. A complete list of structured messages returned by Cortex can be found in the API Reference.

Implemented stronger typing of Helix URI identifier parts

The lack of strong typing for URI identifier parts in the Helix programming model was error prone and created unnecessary work for developers. This has been addressed by adding the ability to specify a <type> value in the <uri-part> definition.

When upgrading:

  • Registering a UriPartTransformer to define a uri-part type no longer works. Instead, the type can now simply be defined in the API definition. If the type is missing in API definition, a default type string is assumed.
  • The generated ResourceIdentifier classes use the <type> in the getter and builder methods for the uri-part. This reduces the need for casting IdentifierPart to an appropriate type. Your static analysis tool will likely complain about unnecessary casts if they were present.

Source code structure changes

The following changes were made to the source code structure:

  • A parent POM was added to the source release package to support building Elastic Path Commerce in a single Maven reactor.
  • The cortex-ep-integration module was renamed to cortex-resources, and now includes source code for Cortex API definitions in the rest-resources-api sub-module.
When upgrading:
  • Rename the cortex-ep-integration module to cortex-resources before merging source changes.

Eliminated Commerce Manager invoker modules

The previous solution to combining Maven and Tycho builds into a single reactor required special invocation of the cm-invoker and ext-cm-invoker modules. These special invoker modules have been eliminated, and new cm-modules and ext-cm-modules modules have been introduced to drive Tycho builds.

The source code has been restructured so that all Tycho modules are now children of cm-modules and ext-cm-modules.

When upgrading:

  • Restructure commerce-manager and extensions/cm modules to match the new source structure before merging source changes.
  • Remove all references to invoker modules from build commands. For example, mvn clean install -f cm-invoker/pom.xml can be replaced with mvn clean install.

Improved support for per-environment configuration

Environment specific configuration files are read at runtime from either the /ep/conf/ or the <user-home>/ep/conf/ directories. Configuration can consist of Cortex configuration files, Environment specific settings, or Ehcache runtime configuration.

The management of these configuration files was improved as follows:
  • CloudOps for AWS and the Pusher deploy files and subdirectories contained in the extensions/database/ext-data/src/main/resources/environments/<environment-name>/files/ directory to the matching runtime environment.
    Note: Requires CloudOps for AWS 2.1 or later
  • All caches backed by Ehcache can be configured in a single file for each environment.
  • Configuration files for the local development environment can be deployed via the Maven using either -Preset-conf or -Pupdate-conf profiles.
  • Developers can override the default cache timeout for their local environment via their Maven settings.xml file.

When upgrading:

  • Consider taking advantage of these capabilities to simplify developer setup and provide cache timeouts that are more appropriate for each environment than the default timeouts.

Improved cache naming consistency

External cache names were updated to improve consistency and accuracy of cache naming.

When upgrading:

  • Spring cache configuration overrides are not impacted and do not need to be changed.
  • External cache configuration files need to be updated to use the new names documented at Core Caching.

Added generation of standalone demo package

A standalone demo of your project can now be created from the extensions/packager module using the command mvn clean install -Pwith-demo-package. This creates a ZIP file containing an embedded H2 database, all webapps, and data population and import/export tools. Instructions on how to run the demo are in the embedded README.txt file.

Added a TORDERSKUPARENT join table

The TORDERSKU parent/child relationship on the PARENT_ITEM_UID column resulted in a full table scan of TORDERSKU whenever an order was read. This was fixed by introducing a new TORDERSKUPARENT join table.

When upgrading:

  • Deploying to a production environment requires a planned system outage. This is not a backward compatible change.
  • If the database contains a large number of orders with bundles, allow additional time for the Liquibase script to populate the TORDERSKUPARENT table.

Renamed VIRTUAL database column to IS_VIRTUAL

MySQL 5.7 added VIRTUAL as a reserved word, which conflicted with the VIRTUAL column in the TMASTERCATEGORY table. This column has been renamed to IS_VIRTUAL for all databases.

When upgrading:

  • There should no be impact unless you have custom code or queries that refer to this column.
  • Deploying to a production environment requires a planned system outage. This is not a backward compatible change.

Upgraded Apache Tomcat to 7.0.81

Apache Tomcat was upgraded to obtain the latest security and bug fixes.

When upgrading:
  • Make sure that Tomcat is updated for all deployment environments.

Upgraded Oracle JDK to 8u144

The Oracle JDK was upgraded to obtain the latest security and but fixes.

When upgrading:
  • Make sure that the JDK is updated for all development, CI and deployment environments.

Upgraded MySQL JDBC driver to 5.1.44

The MySQL JDBC driver was upgraded to obtain the latest security and bug fixes.

When upgrading:

  • You will need to manually download and install the new JDBC driver in your Maven repository as described in the Starting Construction Guide.

Upgraded RxJava to 2.1.0

RxJava was upgraded to version 2.1.0. This is a major version upgrade that introduces:

  • A new Maybe type to represent optional single value results.
  • A new test() operator which can be used to write fluent unit tests. See example at RxJava.

The RxJava upgrade introduced a number of breaking code changes. The changes you will need to make to custom Helix resources depend on your usage of RxJava.

Some of the common changes are listed below. For a complete list of changes in RxJava 2.x see the RxJava wiki.

When upgrading:

  1. Change the groupId for Maven dependencies from io.reactivex to io.reactivex.rxjava2.
  2. Change the package name prefix in Java import statements from import rx. to import io.reactivex. using a global search and replace.
  3. Fix compile errors caused by breaking RxJava changes. Some of the modifications you will need to make are:
    RxJava 1.x Methods RxJava 2.x Methods
    toBlocking().first() blockingFirst()
    toBlocking().single() blockingGet()
    first() firstElement()
    from() fromIterable()
    toSingle() singleElement().toSingle()