Forums - Cache thrashing in bilinear resize op

1 post / 0 new
or Register
Cache thrashing in bilinear resize op
joe_6
Join Date: 5 Oct 20
Posts: 1
Posted: Mon, 2020-10-05 18:39
Top

Hi all,

Apologies if this is the wrong place to post this, but we noticed that in one of our models the bilinear resize operation would occasionally take an exceedingly long time to complete.  The operation typically takes much less than 0.1 ms to complete, but sometimes would take more than 40 ms.  These long execution times happened about 0.01% of the time, but were still frequent enough to have a substantial impact on our application.

We were able to fix the issue by modifying the `resize_bilinear_d32_execute` function.  The original version of this function had the following lines:

```

for(int i =0; i < n_threads; i++)

    nn_os_work_for_vector( nn, runstate.run_all_func, &runstate );

 

// copy the min and max through

tensor_copy( out_min_tensor, in_min_tensor );

tensor_copy( out_max_tensor, in_max_tensor );

nn_sem_wait_n_times( &runstate.done_sem, n_threads);

```

We were able to fix the issue by moving the `tensor_copy` operations to after the threads finish.  The median execution time is slightly longer, but we do not observe excessively long execution times with this change.  We were able to check that when the operation takes a long time to execute there is a much larger number of cache misses, so this seems to be a cache thrashing issue.

Is this a known issue?  Are there other ops that have similar issues?

  • Up0
  • Down0

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.