Vaccine Therapy in Treating Patients With Metastatic Breast Cancer

Inclusion Criteria: * Subjects must have a histologically confirmed diagnosis of metastatic carcinoma of the breast; measurable disease is not required; subjects who are NED are eligible; subjects must have had at least one prior regimen of chemotherapy, immunotherapy, or hormonal therapy prior to entering this study; subjects may have received any number of prior therapies for metastatic disease * Subjects must have an ECOG performance status of 0-1 * WBC \> 2000/mm\^3 * Platelet count \> 100,000/mm\^3 * Serum creatinine =\< 2.0 mg/dl * Serum bilirubin =\< 1.5 mg/dl * SGPT \< 3 times the upper limit of normal * \>= 4 weeks since chemotherapy (\>= 6 weeks for nitrosoureas or mitomycin C), hormonal therapy or radiation therapy; subjects must have recovered from all acute toxicity associated with the prior regimen; subjects receiving concurrent hormonal treatment or local radiation are not eligible * Subjects must be HLA typed if not already previously done (5/10 subjects at the MTD dose level must be HLA A2 positive) * Subjects must not have clinical evidence of altered immune responsiveness or autoimmune syndromes (scleroderma, systemic lupus erythematosus, etc.); subjects must be HIV antibody negative; this treatment may be associated with increased adverse effects for individuals with immune deficiencies, and HIV-associated symptoms preclude accurate assessment of toxicity * Subjects must not have undergone splenectomy * The recombinant vaccinia vaccine should not be administered if the following apply to either recipients or, for at least three weeks after vaccination, their close household contacts: persons with active or a history of extensive eczema or other eczematoid skin disorders; those with other acute, chronic or exfoliative skin conditions (e.g., atopic dermatitis, burns, impetigo, varicella zoster, severe acne or other open rashes or wounds) until condition resolves; pregnant or nursing women; children under 5 years of age; and immunodeficient or immunosuppressed persons (by disease or therapy), including HIV infection; close household contacts are those who share housing or have close physical contact; determination of the severity of these conditions will be made by the investigator or co-investigator * Subjects must not have any other serious medical condition that in the opinion of the investigator is incompatible with the protocol; subjects with active infections requiring antibiotics are not eligible until the infection has cleared and the antibiotics have been stopped for at least 3 days * Subjects must have had prior vaccinia (smallpox) exposure, determined by subject history, medical documentation, or scar characteristic of vaccinia exposure; there must be no history of allergy or untoward reaction to prior vaccinia (smallpox) vaccination * Tumor tissue positive for staining with MAbs DF3 and/or DF3-P or elevated serum CA15-3 (also known as CA27-29); note: this can be done on stored slides, but subject will be responsible for costs if not covered by insurance * Subjects must not have a history of seizures, encephalitis or multiple sclerosis * Subjects must not be allergic to eggs * Women of child-bearing potential must agree to use highly effective contraception or abstinence prior to study entry and for at least 4 weeks after the last vaccination; women who are breast-feeding are not eligible for this study; should a woman become pregnant or suspect she is pregnant while participating in this study, she should inform her treating physician immediately * Signed informed consent * Participants who have been previously treated with vaccinia vectors or MUC1 such as those on protocol T98-0057 (DFPCC # 97-050) are not eligible for this study