Skip to content

Commit a55bc4f

Browse files
nathanchancegregkh
authored andcommitted
staging: rtl8723bs: Avoid memset() in aes_cipher() and aes_decipher()
After commit 6f110a5 ("Disable SLUB_TINY for build testing"), which causes CONFIG_KASAN to be enabled in allmodconfig again, arm64 allmodconfig builds with older versions of clang (15 through 17) show an instance of -Wframe-larger-than (which breaks the build with CONFIG_WERROR=y): drivers/staging/rtl8723bs/core/rtw_security.c:1287:5: error: stack frame size (2208) exceeds limit (2048) in 'rtw_aes_decrypt' [-Werror,-Wframe-larger-than] 1287 | u32 rtw_aes_decrypt(struct adapter *padapter, u8 *precvframe) | ^ This comes from aes_decipher() being inlined in rtw_aes_decrypt(). Running the same build with CONFIG_FRAME_WARN=128 shows aes_cipher() also uses a decent amount of stack, just under the limit of 2048: drivers/staging/rtl8723bs/core/rtw_security.c:864:19: warning: stack frame size (1952) exceeds limit (128) in 'aes_cipher' [-Wframe-larger-than] 864 | static signed int aes_cipher(u8 *key, uint hdrlen, | ^ -Rpass-analysis=stack-frame-layout only shows one large structure on the stack, which is the ctx variable inlined from aes128k128d(). A good number of the other variables come from the additional checks of fortified string routines, which are present in memset(), which both aes_cipher() and aes_decipher() use to initialize some temporary buffers. In this case, since the size is known at compile time, these additional checks should not result in any code generation changes but allmodconfig has several sanitizers enabled, which may make it harder for the compiler to eliminate the compile time checks and the variables that come about from them. The memset() calls are just initializing these buffers to zero, so use '= {}' instead, which is used all over the kernel and does the exact same thing as memset() without the fortify checks, which drops the stack usage of these functions by a few hundred kilobytes. drivers/staging/rtl8723bs/core/rtw_security.c:864:19: warning: stack frame size (1584) exceeds limit (128) in 'aes_cipher' [-Wframe-larger-than] 864 | static signed int aes_cipher(u8 *key, uint hdrlen, | ^ drivers/staging/rtl8723bs/core/rtw_security.c:1271:5: warning: stack frame size (1456) exceeds limit (128) in 'rtw_aes_decrypt' [-Wframe-larger-than] 1271 | u32 rtw_aes_decrypt(struct adapter *padapter, u8 *precvframe) | ^ Cc: [email protected] Fixes: 554c0a3 ("staging: Add rtl8723bs sdio wifi driver") Signed-off-by: Nathan Chancellor <[email protected]> Reviewed-by: Dan Carpenter <[email protected]> Link: https://lore.kernel.org/r/20250609-rtl8723bs-fix-clang-arm64-wflt-v1-1-e2accba43def@kernel.org Signed-off-by: Greg Kroah-Hartman <[email protected]>
1 parent e04c78d commit a55bc4f

File tree

1 file changed

+14
-30
lines changed

1 file changed

+14
-30
lines changed

drivers/staging/rtl8723bs/core/rtw_security.c

Lines changed: 14 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -868,29 +868,21 @@ static signed int aes_cipher(u8 *key, uint hdrlen,
868868
num_blocks, payload_index;
869869

870870
u8 pn_vector[6];
871-
u8 mic_iv[16];
872-
u8 mic_header1[16];
873-
u8 mic_header2[16];
874-
u8 ctr_preload[16];
871+
u8 mic_iv[16] = {};
872+
u8 mic_header1[16] = {};
873+
u8 mic_header2[16] = {};
874+
u8 ctr_preload[16] = {};
875875

876876
/* Intermediate Buffers */
877-
u8 chain_buffer[16];
878-
u8 aes_out[16];
879-
u8 padded_buffer[16];
877+
u8 chain_buffer[16] = {};
878+
u8 aes_out[16] = {};
879+
u8 padded_buffer[16] = {};
880880
u8 mic[8];
881881
uint frtype = GetFrameType(pframe);
882882
uint frsubtype = GetFrameSubType(pframe);
883883

884884
frsubtype = frsubtype>>4;
885885

886-
memset((void *)mic_iv, 0, 16);
887-
memset((void *)mic_header1, 0, 16);
888-
memset((void *)mic_header2, 0, 16);
889-
memset((void *)ctr_preload, 0, 16);
890-
memset((void *)chain_buffer, 0, 16);
891-
memset((void *)aes_out, 0, 16);
892-
memset((void *)padded_buffer, 0, 16);
893-
894886
if ((hdrlen == WLAN_HDR_A3_LEN) || (hdrlen == WLAN_HDR_A3_QOS_LEN))
895887
a4_exists = 0;
896888
else
@@ -1080,30 +1072,22 @@ static signed int aes_decipher(u8 *key, uint hdrlen,
10801072
num_blocks, payload_index;
10811073
signed int res = _SUCCESS;
10821074
u8 pn_vector[6];
1083-
u8 mic_iv[16];
1084-
u8 mic_header1[16];
1085-
u8 mic_header2[16];
1086-
u8 ctr_preload[16];
1075+
u8 mic_iv[16] = {};
1076+
u8 mic_header1[16] = {};
1077+
u8 mic_header2[16] = {};
1078+
u8 ctr_preload[16] = {};
10871079

10881080
/* Intermediate Buffers */
1089-
u8 chain_buffer[16];
1090-
u8 aes_out[16];
1091-
u8 padded_buffer[16];
1081+
u8 chain_buffer[16] = {};
1082+
u8 aes_out[16] = {};
1083+
u8 padded_buffer[16] = {};
10921084
u8 mic[8];
10931085

10941086
uint frtype = GetFrameType(pframe);
10951087
uint frsubtype = GetFrameSubType(pframe);
10961088

10971089
frsubtype = frsubtype>>4;
10981090

1099-
memset((void *)mic_iv, 0, 16);
1100-
memset((void *)mic_header1, 0, 16);
1101-
memset((void *)mic_header2, 0, 16);
1102-
memset((void *)ctr_preload, 0, 16);
1103-
memset((void *)chain_buffer, 0, 16);
1104-
memset((void *)aes_out, 0, 16);
1105-
memset((void *)padded_buffer, 0, 16);
1106-
11071091
/* start to decrypt the payload */
11081092

11091093
num_blocks = (plen-8) / 16; /* plen including LLC, payload_length and mic) */

0 commit comments

Comments
 (0)