<span id="mktg5"></span>

<i id="mktg5"><meter id="mktg5"></meter></i>

        <label id="mktg5"><meter id="mktg5"></meter></label>
        最新文章專題視頻專題問答1問答10問答100問答1000問答2000關鍵字專題1關鍵字專題50關鍵字專題500關鍵字專題1500TAG最新視頻文章推薦1 推薦3 推薦5 推薦7 推薦9 推薦11 推薦13 推薦15 推薦17 推薦19 推薦21 推薦23 推薦25 推薦27 推薦29 推薦31 推薦33 推薦35 推薦37視頻文章20視頻文章30視頻文章40視頻文章50視頻文章60 視頻文章70視頻文章80視頻文章90視頻文章100視頻文章120視頻文章140 視頻2關鍵字專題關鍵字專題tag2tag3文章專題文章專題2文章索引1文章索引2文章索引3文章索引4文章索引5123456789101112131415文章專題3
        問答文章1 問答文章501 問答文章1001 問答文章1501 問答文章2001 問答文章2501 問答文章3001 問答文章3501 問答文章4001 問答文章4501 問答文章5001 問答文章5501 問答文章6001 問答文章6501 問答文章7001 問答文章7501 問答文章8001 問答文章8501 問答文章9001 問答文章9501
        當前位置: 首頁 - 科技 - 知識百科 - 正文

        HBaseMVCCandbuilt-inAtomicOperations

        來源:懂視網 責編:小采 時間:2020-11-09 13:27:53
        文檔

        HBaseMVCCandbuilt-inAtomicOperations

        HBaseMVCCandbuilt-inAtomicOperations:By Lars Hofhansl (This is a follow to my ACID in HBase post from March this year) HBase has a few special atomic operations: checkAndPut, checkAndDelete - these simply check a value of a column as a precondition and then apply the Put or D
        推薦度:
        導讀HBaseMVCCandbuilt-inAtomicOperations:By Lars Hofhansl (This is a follow to my ACID in HBase post from March this year) HBase has a few special atomic operations: checkAndPut, checkAndDelete - these simply check a value of a column as a precondition and then apply the Put or D

        By Lars Hofhansl (This is a follow to my ACID in HBase post from March this year) HBase has a few special atomic operations: checkAndPut, checkAndDelete - these simply check a value of a column as a precondition and then apply the Put or D

        By Lars Hofhansl

        (This is a follow to my ACID in HBase post from March this year)
        HBase has a few special atomic operations:

      1. checkAndPut, checkAndDelete - these simply check a value of a column as a precondition and then apply the Put or Delete if the check succeeded.
      2. Increment, Append - these allow an atomic add to a column value interpreted as an integer, or append to the end of a column, resp.
      3. checkAndPut and checkAndDelete are idempotent in the sense that they can safely be applied multiple time (but note that their outcome might differ when other changes are applied between the retries).

        Increment and Append are not idempotent. They are the only non-repeatable operations in HBase. Increment and Append are also the only operations for which the snapshot isolation provided by HBase's MVCC model is not sufficient... More on that later.

        In turns out that checkAndPut and checkAndDelete are not as atomic as expected (the issue was raised by Gregory and despite me not believing it first he is right - see HBASE-7051).

        A look at the code makes this quite obvious:
        Some of the Put optimizations (HBASE-4528) allow releasing the rowlock before the changes are sync'ed to the WAL. This also requires the lock to be released before the MVCC changes are committed so that changes are not visible to other transaction before they are guaranteed to be durable.
        Another operation (such as checkAndXXX) that acquires the rowlock to make atomic changes may in fact not see current picture of things despite holding the rowlock as there could be still outstanding MVCC changes that only become visible after the row lock was release and re-acquired. So it might operate on stale data. Holding the rowlock is no longer good enough after HBASE-4528.

        Increment and Append have the same issue.

        The fix for this part is relatively simple: We need a "MVCC barrier" of sorts. Instead of completing a single MVCC transaction at the end of the update phase (which will wait for all prior transactions to finish), we just wait a little earlier instead for prior transactions to finish before we start the check or get phase of the atomic operation. This only reduces concurrency slightly, since before the end of the operation we have to await all prior transactions anyway. HBASE-7051 does exactly that for the checkAndXXX operations.

        In addition - as mentioned above - Increment and Append have another issue, they need to be serializable transactions. Snapshot isolation is not good enough.
        For example: If you start with 0 and issue an increment of 1 and another increment of 2 the outcome must always be 3. If both could start with the same start value (a snapshot) the outcome could 1 or 2 depending on which one finishes first.

        Increment and Append currently skirt the issue with an ugly "hack": When they write their changes into the memstore they set the memstoreTS of all new KeyValues to 0! The effect is that they are made visible to other transactions immediately, violating HBase's MVCC. Again see ACID in HBase for an explanation of the memstoreTS.
        This guarantees the correct outcome of concurrent Increment and Append operations, but the visibility to concurrent scanners is not what you expect. An Incremented and Appended value even for partial rows can be become visible to any scanner at any time even though the scanner should see an earlier snapshot of the data.
        Increment and Append are also designed for very high throughput so they actually manipulate HBase's memstore to remove older versions of the columns just modified. Thus you lose the version history of the changes in exchange for avoiding a memstore exploding with version of the many Increments or Appends. This is called "upsert" in HBase. Upsert is nice in that it prevents the memstore being filled will a lot of old value if nobody cares for them. The downside is that is a special operation on the memstore, and hard to get right w.r.t. MVCC. It also does not work with mslab (see this Cloudera blog post for explanation of mslab).

        If you don't care about visibility this is a simple problem, since you can just look through the memstore and remove old values. If you care about MVCC, though, you have to prove first that is safe to remove a KV.

        I tried to fix this almost exactly a year ago (HBASE-4583), but after some discussions with my fellow committers we collectively gave up on that.

        A few days ago I reopened HBASE-4583 and started with a radical patch that gets rid of all upsert-type logic (which set the memstoreTS to 0) and just awaits prior transactions before commencing the Increment/Append. Then I rely on my changes from HBASE-4241 to only flush the versions of columns needed when it is time to flush the memstore to disk. Turns out this is still quite a bit slower (10-15%), since it needs to flush the memstore frequently even thought it leads to mostly empty files. Still that was nice try, as it gets rid of a lot of special code and turns Increment and Append into normal HBase citizens.

        A 2nd less radical version makes upsert MVCC aware.

        That is actually not as easy as it looks. In order to remove a version of a column (a KeyValue) from the memstore you have to prove that is not and will not be seen by any concurrent or future scanner. That means we have to find the earliest readpoint of any scanner and ensure that there is at least one version of the KV older than that smallest readpoint; then we can safely remove any older versions of that KV from the memstore - because any scanner is guaranteed to see a newer version of the KV.
        The "less radical" patch in does exactly that.

        In the end the patch I ended up committed with HBASE-4583 does both:
        If the column family that has the column to be incremented or appended to has VERSIONS set to 1, we perform an MVCC aware upsert added by the patch. If VERSIONS is > 1, we use the usual logic to add a KeyValue to the memstore. So now this behaves as expected in all cases. If multiple versions are requested they are retained and time range queries will work even with Increment and Append; and it also keeps the performance characteristics (mostly) when VERSIONS is set to 1.

        聲明:本網頁內容旨在傳播知識,若有侵權等問題請及時與本網聯系,我們將在第一時間刪除處理。TEL:177 7030 7066 E-MAIL:11247931@qq.com

        文檔

        HBaseMVCCandbuilt-inAtomicOperations

        HBaseMVCCandbuilt-inAtomicOperations:By Lars Hofhansl (This is a follow to my ACID in HBase post from March this year) HBase has a few special atomic operations: checkAndPut, checkAndDelete - these simply check a value of a column as a precondition and then apply the Put or D
        推薦度:
        標簽: in and MVCC
        • 熱門焦點

        最新推薦

        猜你喜歡

        熱門推薦

        專題
        Top
        主站蜘蛛池模板: 国产AⅤ无码专区亚洲AV| 暖暖免费高清日本中文| 亚洲桃色AV无码| 久久不见久久见免费影院www日本| 免费一级毛片免费播放| 久久久久亚洲AV无码去区首| 午夜电影免费观看| 亚洲色少妇熟女11p| 曰皮全部过程视频免费国产30分钟| 亚洲精品无码av片| 国产gav成人免费播放视频| 日韩国产欧美亚洲v片| 亚洲国产婷婷香蕉久久久久久| 免费人成再在线观看网站| 亚洲第一黄片大全| 成人免费一区二区三区| 亚洲人成影院在线| 免费H网站在线观看的| 亚洲日本中文字幕天天更新| 国产小视频在线观看免费| 国产精品偷伦视频免费观看了| 亚洲国产美女精品久久久久∴| 精品无码无人网站免费视频| 精品亚洲成在人线AV无码| 国产美女做a免费视频软件| 国产精品福利片免费看| 久久久久亚洲AV无码永不| 中文字幕影片免费在线观看| 污网站在线观看免费| 亚洲精品午夜无码电影网| 久久久久久精品成人免费图片| 亚洲AV一区二区三区四区| 久久久久无码专区亚洲av| 狼群影院在线观看免费观看直播| 久久亚洲精品国产精品婷婷| 免费人成在线观看网站视频| 久久免费看少妇高潮V片特黄| 亚洲成a人无码亚洲成av无码| 亚洲精品无码AV人在线播放| 全免费毛片在线播放| 一个人晚上在线观看的免费视频|