A Data-Driven Exploration of Arbitration as a Settlement Tool: Are Case Outcomes Affected by the Size of the Claim?
Kluwer Arbitration Blog
May 5, 2019
Please refer to this post as:, ‘A Data-Driven Exploration of Arbitration as a Settlement Tool: Are Case Outcomes Affected by the Size of the Claim?’, Kluwer Arbitration Blog, May 5 2019, http://arbitrationblog.kluwerarbitration.com/2019/05/05/a-data-driven-exploration-of-arbitration-as-a-settlement-tool-are-case-outcomes-affected-by-the-size-of-the-claim/
The growing repository of international commercial arbitration case data maintained by Dispute Resolution Data (DRD) is designed to enable practitioners to use analytical tools, ranging from simple to complex, for gleaning valuable insights into the effectiveness of arbitration as an alternative dispute resolution mechanism. For example, we showed in our first blog post that among the 3,500+ active international arbitration cases in DRD’s database, a sizeable majority result in settlement/withdrawal—and in many cases, this outcome is reached relatively quickly, often within one year following the claim date and before any counter-claims or hearings. Since then, we have published follow-up results that dive continually deeper into the dataset to show how the patterns of arbitration case outcomes can vary depending on case type and other parameters.
Here, we examine the potential effect of the size of the claim, both alone and in conjunction with case type, on the outcomes of international commercial arbitration cases, with a specific focus on whether the case resulted in settlement/withdrawal (as opposed to other outcomes, such as the rendering of an award judgment, dismissal of the case, or administrative closure). For the sake of keeping the analysis relatively simple, we will group the cases into three different ranges of claim amounts corresponding to successively larger orders of magnitude. The first range consists of smaller claims, totaling $1M or less (where M = “million”). The second range includes cases with claim amounts between $1M and $10M, and the third range includes cases with the largest claim amounts (totaling $10M or more).
When looking at an aggregate view across all case types in the DRD database, there appears to be an inverse relationship between the claim amount and the likelihood of reaching settlement or withdrawal. As depicted in Figure 1, the average settlement frequency for cases with the smallest claim amounts ($1M or less) is 60%, with a margin of error of ±2% (computed at a 95% level of confidence). Cases with claim amounts in the range of $1M to $10M had a settlement frequency of 54% ± 3%, and the largest cases ($10M or higher) settled at a frequency of 43% ± 4%.
Figure 1. Estimated frequencies at which settlement/withdrawal is reached for international commercial arbitration cases (since 2005), as computed across all case types from the DRD database, and for successively larger ranges of claim amount ranges. Each measurement is annotated with the DRD Signal Strength, which is closely tied to the size of the sample used in the analysis (thereby reflecting data quantity, not quality.) Claim amounts for international cases have been converted to U.S. Dollars for the purpose of this analysis.
The DRD Signal Strength, introduced in our October 2018 blog post, is our proprietary indicator of the degree of confidence that the data samples used in our analysis are reflective of the corresponding populations of all such international commercial arbitration cases that meet the same criteria. Similar to examples we have shown previously, larger sample sizes yield a higher “signal strength,” and with the relatively large samples analyzed here (1911 cases with claim amounts of $1M or less, 974 cases with claim amounts between $1M and $10M, and 574 cases with claim amounts of $10M or more), the high signal strength — 5 out of 5 for all three claim amount ranges — is not surprising. Consequently, we can be reasonably confident that the true proportion of international commercial arbitration cases reaching settlement appears to decrease with increasing claim size.
The apparent inverse relationship between claim amount and settlement/withdrawal frequency motivates many possible questions, but one must recognize that we are looking at an overall, “aggregate” view of the data. Could a similar relationship be observed for specific case types—or, at the very least, are there any differences in settlement frequency with respect to the claim amount’s order of magnitude?
Figures 2A through 2D illustrates the settlement/withdrawal frequencies, for each of the three claim amount ranges, across four of the most well-represented case types in the DRD database, respectively: commercial contracts, construction, wholesale & retail trade, and hospitality & travel. For at least one of these case types, there appears to be a pattern of decreasing settlement frequency with increasing claim amount, although the “steadily decreasing” behavior observed in the aggregate view (depicted in Figure 1 above) is not as apparent.
For example, smaller cases ($1M or less) in the commercial contracts category (Fig. 2A), appear to settle at a very high rate (71% ± 3%), with the settlement rate dropping off considerably for cases between $1M and $10M (34% ± 12%) and above $10M (39% ± 17%). In addition, we see what appears to be a drop-off in settlement frequency for cases in the hospitality & travel category (Fig. 2D), but this appears to occur only for cases with claim amounts above $10M, and there is a significant margin of error due to the very small number of cases that fall into this claim amount range. For cases in the commercial contracts category, there is no overlap in the margin of error between the groups of cases with the smallest ($1M or less) and largest ($10M or more) claim amounts, suggesting the possibility of a relationship between claim size and case outcome—at least for this particular case type. However, for the other three case types shown (Figs. 2B-2D), the possibility of a relationship between claim amount and settlement frequency is much less clear, owing largely to the significant margin-of-error overlap across all three claim amount ranges.
Figures 2A through 2D. Differences in the estimated proportions of international commercial arbitration cases (since 2005) that result in settlement/withdrawal, for each of four highly represented case types in the DRD database, grouped by claim size range.
The results shown in Figures 2A through 2D suggest that the combination of case type and claim amount could have a potential effect on settlement frequency, but such an effect may only be limited to certain case types. A more complex analysis (at a minimum, a test for statistical significance) would be required to determine the probability that a case’s outcome is potentially dependent on the range in which the case’s claim amount falls. As an example, it can be shown from the results of performing a chi-square (𝛘2) test of independence that for these three ranges of claim amounts (i.e., $1M or less, $1M to $10M, and $10M or higher) and the two outcomes studied herein (i.e., settled/withdrawn vs. not settled/withdrawn), there is a very low, almost negligible probability that reaching an outcome of settlement or withdrawal is independent of the claim amount range—at least for those cases in the commercial contracts category. This appears to be in line with the evidence presented in Figure 2A above, which suggests that—for this case type—there appears to be some relationship between claim amount and settlement frequency.
In contrast, performing a chi-square test of independence for each of the other three case types (construction, wholesale/retail trade, and hospitality & travel) results in a much higher probability of the data observed under the hypothesis that a case’s outcome and the size of its associated claim amount are independent. In other words, we cannot reject the possibility that the size of the claim has little or nothing to do with the outcome of the case, given the data that we currently have available for these three case types. When you look at Figures 2B through 2D above, this should make sense—the proportions of cases reaching settlement/withdrawal for each range of claim amounts appear not to be that different, especially considering the significant overlap in the margins of error across the three claim size ranges.
For space considerations, a detailed walkthrough of the computations involved in this and other tests for significance will be reserved for a future blog post, but of note: a chi-square test based on the data aggregated across all case types (i.e., the data depicted in Figure 1) yields a very low probability that the outcome of a case is independent of its claim amount range. This suggests that performing similar significance tests for other case types (beyond the four case types analyzed herein) could potentially produce results similar to those computed for the commercial contracts category.
DRD continues to grow and strengthen the quantity and quality of international commercial arbitration case data in its repository. In future blog posts, and as noted above, we plan to present at least one tutorial to illustrate the basics of statistical hypothesis tests, such as the chi-square test of independence that was summarized above, and possibly others. We also plan to explore the costs associated with various input variables (including but not limited to claim amount and case type), which can potentially help practitioners to set realistic budgets for international commercial arbitration cases in which they may be involved.