- 
          
- 
                Notifications
    You must be signed in to change notification settings 
- Fork 69
add key-value file! #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Merged
      
      
    
  
     Merged
                    Changes from all commits
      Commits
    
    
            Show all changes
          
          
            19 commits
          
        
        Select commit
          Hold shift + click to select a range
      
      8335a0b
              
                add key-value file!
              
              
                ruclz 156142a
              
                Completed the key-value sort with uint64_t.
              
              
                ruclz 473427c
              
                Merge branch 'main' into main
              
              
                ruclz a05f062
              
                first
              
              
                ruclz fad1753
              
                first
              
              
                ruclz 7821e01
              
                run clang-format and add unit test for key-value sort
              
              
                ruclz ae02b58
              
                add unit test and run clang_format
              
              
                ruclz 5363ba4
              
                Rename Makefile to Makefile.bak2
              
              
                ruclz cd836c7
              
                Rename Makefile.bak to Makefile
              
              
                ruclz 541f914
              
                Delete Makefile.bak2
              
              
                ruclz 8873ea1
              
                Update meson.build
              
              
                ruclz 7ea8230
              
                Creat key-value sort hpp file.
              
              
                ruclz 09ef43a
              
                Creat key-value sort hpp file.
              
              
                ruclz 17b534c
              
                Creat key-value sort hpp file.
              
              
                ruclz 4927fe2
              
                Sorry, I forgot to merge the test_all.cpp in the last commit.
              
              
                ruclz e73aa3f
              
                Delete duplicated code and add benchmarking code of key value sort
              
              
                ruclz b73ba45
              
                Move the get_povit_64bit function and sort_zmm_64bit function to avx5…
              
              
                ruclz c083424
              
                Fix the authors name.
              
              
                ruclz ecc94cf
              
                Change the author of avx512-64bit-common.h file to Rahu.
              
              
                ruclz File filter
Filter by extension
Conversations
          Failed to load comments.   
        
        
          
      Loading
        
  Jump to
        
          Jump to file
        
      
      
          Failed to load files.   
        
        
          
      Loading
        
  Diff view
Diff view
There are no files selected for viewing
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| # Prerequisites | ||
| *.d | ||
|  | ||
| # Compiled Object files | ||
| *.slo | ||
| *.lo | ||
| *.o | ||
| *.obj | ||
|  | ||
| # Precompiled Headers | ||
| *.gch | ||
| *.pch | ||
|  | ||
| # Compiled Dynamic libraries | ||
| *.so | ||
| *.dylib | ||
| *.dll | ||
|  | ||
| # Fortran module files | ||
| *.mod | ||
| *.smod | ||
|  | ||
| # Compiled Static libraries | ||
| *.lai | ||
| *.la | ||
| *.a | ||
| *.lib | ||
|  | ||
| # Executables | ||
| *.exe | ||
| *.out | ||
| *.app | ||
|  | ||
| **/.vscode | ||
|  | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
      
      Oops, something went wrong.
        
    
  
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if std::sort is a fair comparison. The way the key-values are arranged in memory for
std::vector<sorted_t<K, V>>is very different when compared two distinct vectorsstd::vector<K>, std::vector<V>. I would think this can alter performance significantly because you can no longer load continuous chunk of memory into ZMM registers. Does Oceanbase have two separate vectors for key and values (I assume they do cos that's the way you implemented the sort). How does Oceanbase currently sort them?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ObStoreRow is the base unit to do the sort in Oceanbase. The key and value are all ObObject class type in the ObStoreRow. There is no sperate vector for key and value. The ObStoreRow is passed to the std::sort in Oceanbase. std::sort first get the key type from ObObject and then do the sort work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To enable the AVX512 sort, we need extract the key from the ObStoreRow and then pass the key and ObStoreRow as the value to avx512_sort(key, value) like API. That's way we need the key-value interface