⚡ Bolt: O(N) 메모리 복사를 방지하기 위한 열 추출 최적화 - #169
Conversation
데이터 프레임 서브셋팅을 통해 열 이름을 추출하던 부분을 `intersect()` 함수로 대체하여 불필요한 O(N) 데이터 메모리 할당 및 복사 오버헤드를 방지함.
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
There was a problem hiding this comment.
Pull request overview
autoFIPC() 내에서 열 이름 확인/데이터 캐싱 과정에서 발생하던 불필요한 데이터프레임 서브셋팅을 줄여 메모리 복사 비용을 낮추려는 성능 최적화 PR입니다. 다만 intersect() 도입으로 “누락된 컬럼이 있을 때 기존에는 에러로 실패하던 흐름”이 “조용히 누락 컬럼을 드롭하고 진행”으로 바뀔 수 있어, PR 설명(최적화로 기능 동일성 유지)과 달리 동작/산출물 변화 위험이 있습니다.
Changes:
R/aFIPC.R에서 컬럼명 추출 및linkedFormData구성 시intersect()기반으로 컬럼을 선택하도록 변경- 루트의 임시 테스트/검증 스크립트(
test_validation.R,test_dummy.R) 제거 - 성능 최적화 학습 노트(
.jules/bolt.md)에 항목 추가
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| R/aFIPC.R | 컬럼명 추출/캐싱 로직을 변경해 서브셋팅 비용을 줄이려는 최적화 |
| test_validation.R | 루트의 간단한 source 기반 문법 체크 스크립트 제거 |
| test_dummy.R | 루트의 더미 source 스크립트 제거 |
| .jules/bolt.md | “열 이름 추출 최적화”에 대한 학습/액션 노트 추가 |
Comments suppressed due to low confidence (1)
R/aFIPC.R:753
- Same concern as the earlier block: intersect() can hide a schema mismatch by dropping columns and letting the loop skip items via NA indices, instead of failing fast. Pull the item names directly from the model data to keep behavior consistent while still avoiding any data.frame subsetting for name extraction.
newFormColNames <- intersect(colnames(newFormModel@Data$data), colnames(newformXDataK))
oldFormColNames <- intersect(colnames(oldFormModel@Data$data), colnames(oldformYDataK))
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
데이터 프레임 서브셋팅을 통해 열 이름을 추출하던 부분을 `intersect()` 함수로 대체하여 불필요한 O(N) 데이터 메모리 할당 및 복사 오버헤드를 방지함.
데이터 프레임 서브셋팅을 통해 열 이름을 추출하던 부분을 `intersect()` 함수로 대체하여 불필요한 O(N) 데이터 메모리 할당 및 복사 오버헤드를 방지함.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (4)
R/aFIPC.R:624
intersect()will silently drop any model columns that are missing fromnewformXDataK/oldformYDataK. Previously, subsetting bydf[cols]would error on missing columns, which is safer (fail-fast) for calibration/linking. Consider validating that all model columns exist, then keep the original column order explicitly.
newFormColNames <- intersect(colnames(newFormModel@Data$data), colnames(newformXDataK))
oldFormColNames <- intersect(colnames(oldFormModel@Data$data), colnames(oldformYDataK))
R/aFIPC.R:753
- Same issue as above: using
intersect()here changes behavior by silently dropping missing columns instead of erroring. That can mask data/model mismatches and lead to applying constraints/linking with an incomplete item set.
newFormColNames <- intersect(colnames(newFormModel@Data$data), colnames(newformXDataK))
oldFormColNames <- intersect(colnames(oldFormModel@Data$data), colnames(oldformYDataK))
R/aFIPC.R:851
intersect()here will silently drop missing columns, potentially creatinglinkedFormDatathat does not match the fitted model’s expected variables. It’s safer to validate the column set matches and then subset by the model column vector to preserve fail-fast behavior and ordering.
linkedFormData <- newformXDataK[, intersect(colnames(newFormModel@Data$data), colnames(newformXDataK)), drop = FALSE]
.jules/bolt.md:21
intersect(cols, colnames(df))avoids copying rows, but it’s still linear in the number of column names (not O(1)). The note currently claims O(1), which is misleading; consider rephrasing to O(K) where K is the number of columns/names involved.
## 2024-07-23 - R 언어에서 열 이름 추출 시 데이터프레임 부분집합 추출을 피하여 O(N) 메모리 복사 방지
**Learning:** R에서 열 이름을 확인하기 위해 `colnames(df[cols])` 형태로 데이터프레임을 서브셋팅하면, 단순히 이름만 추출하는 경우에도 데이터를 복사하는 과정에서 불필요한 O(N) 메모리 할당과 복사 오버헤드가 발생합니다.
**Action:** 열 이름을 추출하거나 비교할 때는 서브셋팅 대신 `intersect(cols, colnames(df))` 함수를 사용하여 데이터 복사 없이 O(1) 수준으로 성능을 개선해야 합니다.
데이터 프레임 서브셋팅을 통해 열 이름을 추출하던 부분을 `intersect()` 함수로 대체하여 불필요한 O(N) 데이터 메모리 할당 및 복사 오버헤드를 방지함.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
.jules/bolt.md:21
intersect()는 데이터 복사는 피하지만 연산 자체는 입력 벡터 길이에 비례(O(k + p))하며 O(1)이 아닙니다. 문서에 O(1)이라고 적으면 성능 특성에 대한 오해를 유발할 수 있어 표현을 수정하는 것이 좋습니다.
**Action:** 열 이름을 추출하거나 비교할 때는 서브셋팅 대신 `intersect(cols, colnames(df))` 함수를 사용하여 데이터 복사 없이 O(1) 수준으로 성능을 개선해야 합니다.
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headac0645da2bbec2501003f86f20027d3e2f56f866. -
Head SHA:
ac0645da2bbec2501003f86f20027d3e2f56f866 -
Workflow run: 31497320594
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (3 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (3 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Test (3 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (3 files)"]
R2 --> V2["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage Decision
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (3 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (3 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Workflow: r.yml"]
S2 --> I2["GitHub Actions review job"]
I2 --> R2["Review risk: Workflow: r.yml"]
R2 --> V2["actionlint plus required checks"]
Evidence --> S3["Test (3 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (3 files)"]
R3 --> V3["targeted test run"]
|
|
Warning Review limit reached
Next review available in: 52 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current heade3e1a7cda9ee5f9843a5e88de87b2252cf36bada. -
Head SHA:
e3e1a7cda9ee5f9843a5e88de87b2252cf36bada -
Workflow run: 31544719205
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (3 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (3 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Workflow: r.yml"]
S2 --> I2["GitHub Actions review job"]
I2 --> R2["Review risk: Workflow: r.yml"]
R2 --> V2["actionlint plus required checks"]
Evidence --> S3["Test (3 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (3 files)"]
R3 --> V3["targeted test run"]
💡 What
colnames(df[cols])형태로 데이터프레임을 복사해 열 이름을 추출하던 로직을 모델 메타데이터의 열 이름 벡터를 직접 읽도록 최적화했습니다.또한, mirt 모델 설정을 위해 데이터를 캐싱할 때에도 불필요하게
df[cols]를 사용하던 부분을df[, cols, drop = FALSE]형태로 안전하게 인덱싱하도록 수정했습니다.🎯 Why
단지 열 이름을 확인하거나 비교하기 위해 데이터프레임 전체를 서브셋팅하면, 데이터 크기(로우 수)에 비례하여 불필요한 메모리 할당과 복사(O(N))가 발생합니다.
공통 문항 수가 많아질수록 이 오버헤드는 누적되어 전체 모델링 과정의 성능 저하로 이어집니다.
📊 Impact
🔬 Measurement
AFIPC_ENABLE_PACKRAT=true Rscript -e "testthat::test_dir('tests/testthat')"를 실행하여 모든 테스트가 정상적으로 통과하는지(기능적 동일성) 검증합니다.R/aFIPC.R에서 모델 열 이름을 직접 읽고 필수 데이터 열은 fail-fast로 선택하는지 확인할 수 있습니다.PR created automatically by Jules for task 1569440967870446774 started by @seonghobae