Skip to content

Commit 585f838

Browse files
headtr1ckclaude
andcommitted
Add performance comparison of vectorize=True and vectorize=[dims]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 050d115 commit 585f838

1 file changed

Lines changed: 83 additions & 0 deletions

File tree

‎advanced/apply_ufunc/automatic-vectorizing-numpy.ipynb‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -435,6 +435,89 @@
435435
"metadata": {
436436
"tags": []
437437
},
438+
"source": [
439+
"### Performance\n",
440+
"\n",
441+
"The difference matters for larger arrays. Let's compare both approaches on the full dataset with 2920 time steps and 53 longitudes, without the `print` statements:"
442+
]
443+
},
444+
{
445+
"cell_type": "code",
446+
"execution_count": null,
447+
"id": "20",
448+
"metadata": {
449+
"tags": []
450+
},
451+
"outputs": [],
452+
"source": [
453+
"air_full = xr.tutorial.load_dataset(\"air_temperature\").air.sortby(\"lat\")\n",
454+
"newlat_full = xr.DataArray(\n",
455+
" np.linspace(20, 70, 100) + np.linspace(0, 3, air_full.sizes[\"time\"])[:, np.newaxis],\n",
456+
" dims=(\"time\", \"lat\"),\n",
457+
")\n",
458+
"\n",
459+
"\n",
460+
"def interp_np(xi, x, data):\n",
461+
" return np.interp(xi, x, data)\n",
462+
"\n",
463+
"\n",
464+
"def interp_batched(xi, x, data):\n",
465+
" return make_interp_spline(x, data, k=1, axis=-1)(xi.reshape(xi.shape[-1]))\n",
466+
"\n",
467+
"\n",
468+
"def interp_full(func, vectorize):\n",
469+
" return xr.apply_ufunc(\n",
470+
" func,\n",
471+
" newlat_full,\n",
472+
" air_full.lat,\n",
473+
" air_full,\n",
474+
" input_core_dims=[[\"lat\"], [\"lat\"], [\"lat\"]],\n",
475+
" output_core_dims=[[\"lat\"]],\n",
476+
" exclude_dims={\"lat\"},\n",
477+
" vectorize=vectorize,\n",
478+
" )"
479+
]
480+
},
481+
{
482+
"cell_type": "code",
483+
"execution_count": null,
484+
"id": "21",
485+
"metadata": {
486+
"tags": []
487+
},
488+
"outputs": [],
489+
"source": [
490+
"%timeit -n 1 -r 3 interp_full(interp_np, vectorize=True)"
491+
]
492+
},
493+
{
494+
"cell_type": "code",
495+
"execution_count": null,
496+
"id": "22",
497+
"metadata": {
498+
"tags": []
499+
},
500+
"outputs": [],
501+
"source": [
502+
"%timeit -n 1 -r 3 interp_full(interp_batched, vectorize=[\"time\"])"
503+
]
504+
},
505+
{
506+
"cell_type": "markdown",
507+
"id": "23",
508+
"metadata": {
509+
"tags": []
510+
},
511+
"source": [
512+
"Looping over `time` only is about 3 times faster: `interp_batched` is called 2920 times instead of 2920 × 53 = 154,760 times, even though a single call of `make_interp_spline` is slower than a single call of `np.interp`."
513+
]
514+
},
515+
{
516+
"cell_type": "markdown",
517+
"id": "24",
518+
"metadata": {
519+
"tags": []
520+
},
438521
"source": [
439522
"## Conclusion\n",
440523
"This is why `apply_ufunc` is so convenient; it takes care of a lot of code necessary to apply functions that consume and produce numpy arrays to xarray objects.\n",

0 commit comments

Comments
 (0)